static-web-server/benchmarks

A benchmark suite which measures the requests per second and latency on average for several web servers.

Static Web Servers Benchmarks

A benchmark suite which measures the requests per second and latency on average for several web servers.

Motivation and context

We wanted to prepare some benchmarks to have a general idea about the SWS performance path over time as well as an excuse to improve/optimize SWS in that regard.

Important Remarks

• Note that this is a rough reference about some particular performance aspects (requests per second and latency on average) so take it at your own risk.
• Be also aware that this benchmark disables in-memory cache for some servers intentionally since SWS doesn't provide such a feature yet.
  In the case of some web servers providing that by default then it gets disabled explicitly. See for example binserve's author comment in this context.
• Have also in mind that some web servers follow approaches like Nignx's hybrid disk‑and‑memory cache strategy which could work in its favor.

Benchmarks history

• 2023-04
• 2022-10

Perform the benchmarks

Change URL env with the corresponding server URL to load test and generate the wrk metrics.

# NOTE: the load test is per server
SERVER="sws" URL="http://localhost" make wrk
# wrk -c 500 -t 12 -d 10s --latency http://localhost

After that, wrk will save two files (JSON/CSV) on disk with the resulting metrics.

For more details about settings see Makefile

System used

• OS: Arch Linux
• Kernel: 6.1.11-arch1-1 (64 bits)
• Processor: 4 × Intel® Core™ i7-6500U
• RAM: 8 GiB

Contributions

Of course, if you find any issue or remark/improvement to do please don't hesitate to file an issue or send a Pull request.

benchmarks's Issues

Any resource usage benchmark?

It would be interesting to also see how those servers fare regarding memory and (average?) CPU usage.
Personally, I'm running some very low-resource servers (well, relative to today's standards) for my friends and I, say a netbook with an Atom N270 and 2GB of RAM.

Add 2024 benhmarks

@joseluisq Are there plans to update the benchmarks? The versions its comparing to are very old by now.

Add lwan to benchmarks

https://github.com/lpereira/lwan is extremely fast.