tchaloupka/httpbench

Simple http benchmark

httpbench's Issues

Dlang/Photon - probably not a single core/thread test as it seemed to be

Check it's behavior - https://github.com/DmitryOlshansky/photon/blob/master/src/photon/linux/core.d#L401

D closed source project http performance

hi @tchaloupka

I build a http framework with D, this projects is not open source yet. Right now I can provide binary to test.

https://github.com/changloong/libuv/releases/download/v1.39.0/tests_http_server_linux_x64
https://github.com/changloong/libuv/releases/download/v1.39.0/tests_http_server_macOS_x64

This is testing http server listen on 0.0.0.0:8080, only support 1 thread, the results from 7700K:

wrk -c200 -t1 -d3 http://127.0.0.1:8080/hello
Running 3s test @ http://127.0.0.1:8080/hello
  1 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   588.80us  154.95us   1.45ms   90.33%
    Req/Sec   203.29k     4.55k  210.64k    80.00%
  606416 requests in 3.00s, 52.05MB read
Requests/sec: 201858.22
Transfer/sec:     17.33MB

I am curious the results from your testing server. Your can run it with unprivileged user or from docker.

New http server in D

Would be interesting to see the updated run (some frameworks could improve their stats), moreover in D community new http server has been created: https://github.com/trikko/serverino

Most projects do not Date keywords in the information header.

Test project code need have Date: Wed, 17 Apr 2013 12:00:00 GMT, but some project not have.

httpbench/dlang/eventcore/http-server.d

Lines 76 to 84 in 0e66935

	auto reply = cast(const(ubyte)[])(
	"HTTP/1.1 200 OK\r\n"
	~ "Server: eventcore_0123456789012345678901234567890123456789\r\n"
	~ "X-Test: 01234567890123456789\r\n"
	~ "Content-Length: 13\r\n"
	~ "Content-Type: text/plain\r\n"
	~ "Connection: keep-alive\r\n"
	~ "\r\n"
	~ "Hello, World!");

Add peak used RSS memory to the results

Might be interesting to see the differences too.

Add generic prefork runner for singleCore tests to be comparable with multiCore ones

Basic idea is to just use SO_REUSEPORT on the listening sockets and add prefork runner that'll fork multiple singlecore test subprocesses and manage them (signals).
See ie https://tech.flipkart.com/linux-tcp-so-reuseport-usage-and-implementation-6bfbf642885a for details (also mentions SO_REUSEPORT should be more effective than listen/fork/accept on the same socket.

This can be interesting to test internal userspace platform schedulers against Linux loadbalancer. (ie GO fasthttp test on techempower is faster when used with their prefork package).

Add switch to test HTTP pipelining

https://en.wikipedia.org/wiki/HTTP_pipelining
See: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/toolset/wrk/pipeline.lua

HTTP/1.1 servers should support this.

Build error with make all

2020-09-21 20:25:45 ~/code/d/httpbench (master)
$ uname -a
Linux AIM-ES 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
2020-09-21 20:29:35 ~/code/d/httpbench (master)
$ ldc2 --version
LDC - the LLVM D compiler (1.17.0):
  based on DMD v2.087.1 and LLVM 8.0.1
  built with LDC - the LLVM D compiler (1.17.0)
  Default target: x86_64-unknown-linux-gnu
  Host CPU: haswell
  http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    mips       - MIPS (32-bit big endian)
    mips64     - MIPS (64-bit big endian)
    mips64el   - MIPS (64-bit little endian)
    mipsel     - MIPS (32-bit little endian)
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    riscv32    - 32-bit RISC-V
    riscv64    - 64-bit RISC-V
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    wasm32     - WebAssembly 32-bit
    wasm64     - WebAssembly 64-bit
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
2020-09-21 20:29:42 ~/code/d/httpbench (master)
$ make all
_suite/runner.d bench --type all
_suite/runner.d(494,27): Error: undefined identifier net in module std, did you mean overloadset get?
/opt/dev-setup/ldc2-1.17.0-linux-x86_64/bin/ldc2 failed with exit code 1.
make: *** [Makefile:25: all] Error 2
2020-09-21 20:30:07 ~/code/d/httpbench (master)
$

Add command to output responses from all tests

Just to check the size and content for each tests output

Techempower

Are these benchmarks reflected in those at techempower?

It seems to me stuff is missing there

https://www.techempower.com/benchmarks/