New config to adjust timeouts between NGINX and PHP-FPM (fix PHP zombie process)

[diogoscota]

New config to adjust timeouts between NGINX and PHP-FPM (fix PHP zombie process)

[bukka]

But how can PHP know that nginx already answered the client with 504 response? Please note that FPM does not know the nginx timeout or even when the request is sent. It would require providing some fcgi env with time and some extra configuration in FPM for time limit (this should most likely match the timeout set in nginx). It would also require some configuration for the variable name as this is non standard. Is that what you asking for? I can see the use case but something like this would require some extra configuration on both parts (nginx and FPM) so not sure how much used this would be.

[diogoscota]

Hello, bukka!

In this case, no configuration is needed on NGINX. What needs to be adjusted is the PHP-FPM timeout, considering the time from when the process arrives in the backlog, not from when the Worker picks up the request from the backlog for processing (disregarding the time in the backlog).

This issue is critical because once PHP “loses synchronization with NGINX,” clients will only start receiving responses from PHP when the backlog queue decreases.

In the meantime, PHP will process several requests uselessly.

I hope this helps!

[bukka]

The problem is that PHP-FPM does not know the time from when process arrives in the backlog. This is managed by kernel and it's in the buffer so not sure how could we get such info. Do you have any idea?

[bukka]

I don't this this is possible in the current architecture as it's read one by one record from the socket so we would need to pre-read all incoming records first and then distribute them between children but that's completely different design and not something that can be really done in FPM - we would need a new SAPI for that.

[diogoscota]

Hello, Bukka,

I understand, thank you for the information. Based on this, I can conclude that PHP-FPM has a significant flaw in handling HTTP requests when connected to a reverse proxy (such as NGINX).

In a scenario of DDoS attacks or resource shortages, PHP-FPM will become indifferent to the clients' purpose (which is to receive a timely response).

I solved the problem in a simple way: I configured an additional parameter in NGINX to inform PHP of the exact time the request arrived at the server.

This way, when the PHP script executes, it first checks if it is already delayed. If it is, it terminates the process immediately, saving a lot of resources.

I have a server where I conducted various load tests to identify and solve the problem (I can show you if you want). By configuring this option, the server became completely resilient to the issue.

One detail, it would be easier if the $_SERVER['REQUEST_TIME'] variable actually reflected the time the request arrived at the server, but it doesn’t. It marks the exact time the script started executing, so it’s not useful in this case. I had to rely on NGINX to pass this information.

[diogoscota]

@bukka, I understand that a configuration like this should come by default in PHP-FPM. It would save a lot of headaches, especially for those who don't have the technical knowledge to solve such problems.

In an ideal world, it would be great if PHP-FPM had a (boolean) configuration to consider the entire request time for timeouts. This would be wonderful and perhaps even enabled by default.

Since we apparently don't have this option, my suggestion would be to have a configuration to enable PHP-FPM to consider the request arrival time at the server through a fastcgi_param with a standard name, like REQUEST_START_TIME. This is a solution similar to the one I implemented, but in this case, it would be more efficient as it would be native to PHP-FPM.

Basically, if this variable were present, PHP-FPM would internally validate if the REQUEST_START_TIME parameter was indeed passed and use it to calculate timeouts.

Thank you for your attention and sorry for the lengthy text, but I thought it was important to share.

[bukka]

I think this feature request makes sense.

[bukka]

I think we should start with configurable env maybe with some default value.

It could be actually interesting idea to introduce mode that would distribute fcgi requests between children. It would need to be optional but could potentially allow multiplexing. It's not a small job to do though.

[diogoscota]

Hi @bukka ,

You mentioned the idea of distributing fcgi requests between child processes. Are you suggesting eliminating the main process and distributing requests directly among the child processes? If that's the case, I believe it would be a significant task and might depend on additional configurations in NGINX, which wouldn't be backward compatible with most installations. This could be a problem. (please correct me if I misunderstood)

If there's a way to do this without breaking compatibility, it might be a good idea.

Regarding my suggestion (configuration to fetch the initial request time through a fast_cgi parameter), I think it would be a good idea and relatively quick to implement. In this case, the process receiving the request could calculate the total request time using the parameter and reject it without starting the processing, saving resources.

But I must reiterate, the ideal solution would be for PHP itself to identify when the process entered the backlog. In this case, no additional configuration in NGINX would be necessary.

(If you know someone, tag a person directly involved with PHP-FPM so we can enhance the solution.)