Add "without keep alive" metric

[KostyaTretyak]

Add "without keep alive" metric

[joanhey]

If you use http1.1 or http2.0, the keep alive is automatic.
In reality we can remove it from the wrk headers, and still it will use keep alive.

An SPA is very difficult that only use 1 call to the server.

[KostyaTretyak]

f you use http1.1 or http2.0, the keep alive is automatic.

Yes, so what? If "keep alive" is not used, it does not matter if this feature is enabled or not.

An SPA is very difficult that only use 1 call to the server.

As a developer who has been professionally developing SPAs since 2014, I can assure you that the complexity of SPAs has no effect on requests to the backend. In SPA, more often than not, a single request to the backend is used when changing the URL. Even if the SPA doesn't use GraphQL, the frontend is still typically send one request at a time.

[fakeshadow]

Modern http favor tcp/udp connection reuse for good reason. Disabling keep alive is neither practical nor meaningful(perf test wise).

[KostyaTretyak]

Disabling keep alive is neither practical nor meaningful

Yes, tell that to the users who use GraphQL that they are "not modern" and instead of one query they need to use dozens of queries.

[joanhey]

I think that you are misunderstanding the benefits of Keep Alive.
It's the same if they only need 1 request to the server, the next request will be slower.

Remove Keep Alive is like returning 30 years in the past.

And worst if you are using SSL.

[KostyaTretyak]

The Keep Alive feature is used, but this is done by front-end servers, which usually run nginx, apache, etc. They handle static files, including SSL.

[joanhey]

Sorry, but it isn't so.
If you connect your server app via proxy with nginx, apache,... without Keep Alive. The performance will be terrible.

[KostyaTretyak]

@joanhey, what are you talking about? Describe a specific real-life scenario where the client is an SPA that is written in React or Angular.

From my many years of practice, all static requests are served by nginx with the "keep alive" feature enabled, and the backend receives requests only for dynamic requests. As I wrote above, probably 90% of dynamic requests do not use "keep alive".

[joanhey]

https://www.nginx.com/blog/http-keepalives-and-web-performance/

https://www.nginx.com/blog/avoiding-top-10-nginx-configuration-mistakes/#no-keepalives

It's the same if it's SPA or a normal app.

[KostyaTretyak]

@joanhey, I'm not the first time I see how you actually bring something that is not an argument in place of arguments. And now you've done it again. I know very well what "keep alive" is, why are you giving me a link to an article that describes what it is and how to enable it?

[joanhey]

I send you 2 articles that explain why you need to use it.

If you can't understand it, I can't give you more information.

[KostyaTretyak]

If you can't understand it, I can't give you more information.

Please don't be rude to me, because I can do it too and our discussion will not go in a constructive direction.

[bhauer]

Single Page Applications (which are very popular now) often use two web servers: one for static files, one for dynamic requests. Benchmarks are usually run for frameworks with dynamic requests, and probably in 90% of cases, such a feature as "keep alive" is not used (in particular because of use a single GraphQL query).

In a typical back-end deployment, an HTTP reverse proxy (e.g., nginx, HAProxy, Traefik) is positioned between clients and the back-end application. Whether or not HTTP KeepAlive is used between the client and the HTTP reverse proxy, it is extremely common to have HTTP KeepAlive enabled between the proxy and the back-end. This is precisely for the performance reasons you cite, namely, closing the proxy server's connection to the back-end after each front-end request would be a significant performance tax on both servers. Since this project measures the maximum throughput of back-end technologies, we specify the use of KeepAlive to be consistent with this common configuration.

(Sidebar: it's acknowledged that there are some back-end technologies that subsume the typical proxy server into the back-end itself. In this configuration, in many cases it remains marginally beneficial to enable KeepAlive to benefit scenarios where a client may make multiple requests during the keep-alive window.)

Can you elaborate on the specific architectural configuration you anticipate would be represented by testing with KeepAlive disabled?

[KostyaTretyak]

Whether or not HTTP KeepAlive is used between the client and the HTTP reverse proxy, it is extremely common to have HTTP KeepAlive enabled between the proxy and the back-end.

Show me the part where I deny that. When I say that 90% of SPAs requests do not use Keep Alive - that means that 10% of requests use this feature. So I'm saying that in real life Keep Alive is enabled on the frontend and backend servers. But we're talking about benchmarks on TechEmpower that completely ignore the ability of servers to work when Keep Alive is not used (regardless of whether it is enabled by the server or not). So, to simulate work without Keep Alive, we just need to ask the backend servers to close the connection after processing each request.

Can you elaborate on the specific architectural configuration you anticipate would be represented by testing with KeepAlive disabled?

wrk -H 'Connection: close' # ...

[bhauer]

Show me the part where I deny that.

There is no need for defensiveness, as I am not accusing you of anything. I am trying to understand the motivation for what you're requesting.

But we're talking about benchmarks on TechEmpower that completely ignore the ability of servers to work when Keep Alive is not used (regardless of whether it is enabled by the server or not).

I'll rephrase my inquiry from the previous message: Aside from purely academic curiosity, what value do you expect could be gleaned from benchmarks run with HTTP KeepAlive disabled?

What deployment architecture would such a test approximate? I have not personally deployed or seen back-end deployments where the reverse proxy and back-end are not using kept-alive connections. In other words, to my understanding, a test without HTTP KeepAlive on the back-end would not be representative of any real-world scenarios. (I'll reiterate that I understand your point that SPAs will frequently use non-KeepAlive connections to the back-end's reverse proxy, but that is an independent matter.)

As much as this project and all of the included test types are synthetic, it nevertheless is a goal that the tests serve as high-water marks of performance for real-world applications. If there is a real-world scenario where back-end servers would be receiving requests without reusing connections from the reverse proxy, perhaps others on the project could be convinced of the value of such testing.