spencergibb/spring-cloud-gateway-bench

Simple benchmark comparing zuul and spring cloud gateway

Spring Cloud Gateway Benchmark

TL;DR

Proxy	Avg Latency	Avg Req/Sec/Thread
gateway	6.61ms	3.24k
linkered	7.62ms	2.82k
zuul	12.56ms	2.09k
none	2.09ms	11.77k

Terminal 1 (simple webserver)

cd static
./webserver # or ./webserver.darwin-amd64 on a mac

Terminal 2 (zuul)

cd zuul
./mvnw clean package
java -jar target/zuul-0.0.1-SNAPSHOT.jar 

Terminal 3 (gateway)

cd gateway
./mvnw clean package
java -jar target/gateway-0.0.1-SNAPSHOT.jar 

Terminal 4 (linkerd)

cd linkerd
java -jar linkerd-1.3.4.jar linkerd.yaml

Terminal N (wrk)

install wrk

Ubuntu: sudo apt install wrk

Mac: brew install wrk

NOTE: run each one multiple times to warm up jvm

Gateway bench (8082)

$ wrk -t 10 -c 200 -d 30s http://localhost:8082/hello.txt
Running 30s test @ http://localhost:8082/hello.txt
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.61ms    4.71ms  49.59ms   69.36%
    Req/Sec     3.24k   278.42     9.02k    75.89%
  969489 requests in 30.10s, 175.67MB read
Requests/sec:  32213.38
Transfer/sec:      5.84MB

zuul bench (8081)

~% wrk -t 10 -c 200 -d 30s http://localhost:8081/hello.txt
Running 30s test @ http://localhost:8081/hello.txt
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.56ms   13.35ms 195.11ms   86.33%
    Req/Sec     2.09k   215.10     4.28k    71.81%
  625781 requests in 30.09s, 123.05MB read
Requests/sec:  20800.13
Transfer/sec:      4.09MB

linkerd bench (4140)

~% wrk -H "Host: web" -t 10 -c 200 -d 30s http://localhost:4140/hello.txt
Running 30s test @ http://localhost:4140/hello.txt
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.62ms    5.45ms  53.51ms   69.82%
    Req/Sec     2.82k   184.58     4.11k    72.17%
  843418 requests in 30.07s, 186.61MB read
Requests/sec:  28050.76
Transfer/sec:      6.21MB

no proxy bench (8000)

~% wrk -t 10 -c 200 -d 30s http://localhost:8000/hello.txt
Running 30s test @ http://localhost:8000/hello.txt
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.09ms    2.07ms  28.37ms   85.89%
    Req/Sec    11.77k     2.07k   45.46k    70.97%
  3516807 requests in 30.10s, 637.24MB read
Requests/sec: 116841.15
Transfer/sec:     21.17MB

spring-cloud-gateway-bench's Issues

why it's so slow on my container?

Could you give me some hints？
my container: cpu cores 4, memory 4 gb

wrk -t12 -c400 -d 30s http://localhost:8000/hello.txt

Running 30s test @ http://localhost:8000/hello.txt
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 53.81ms 63.06ms 701.57ms 88.71%
Req/Sec 0.90k 301.91 5.94k 80.05%
320851 requests in 30.04s, 45.29MB read
Requests/sec: 10682.54
Transfer/sec: 1.51MB

how about zuul2 vs springcloud gateway?

rt

thanks。

Zuul 2 has been released

Hi, zuul 2.1 has been released and it offers a non blocking proxy, would you like to check it out and update the readme.md file?

test machine

can you provide the test machine configuration? such as
CPU core number
Memory size
...

gateway-0.0.1-SNAPSHOT.jar can't run

I clone spring-cloud-gateway-bench repository, and package gateway runnable jar, but when i run it, i got this:
java.lang.IllegalStateException: Failed to introspect Class [org.springframework.cloud.gateway.config.GatewayAutoConfiguration$NettyConfiguration] from ClassLoader [org.springframework.boot.loader.LaunchedURLClassLoader@20ad9418]
at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:686) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:583) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:568) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:626) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688) ~[na:1.8.0_192]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:721) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:662) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:630) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1518) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:507) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:477) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:598) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:590) ~[spring-beans-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.getBeansOfType(AbstractApplicationContext.java:1204) ~[spring-context-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
at org.springframework.boot.SpringApplication.getExitCodeFromMappedException(SpringApplication.java:905) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.getExitCodeFromException(SpringApplication.java:891) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.handleExitCode(SpringApplication.java:877) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:826) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:327) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
at com.example.gateway.GatewayApplication.main(GatewayApplication.java:21) [classes!/:0.0.1-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_192]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_192]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_192]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_192]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gateway-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gateway-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gateway-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gateway-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: java.lang.NoClassDefFoundError: reactor/netty/http/client/HttpResponseDecoderSpec
at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_192]
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_192]
at java.lang.Class.getDeclaredMethods(Class.java:1975) ~[na:1.8.0_192]
at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:668) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
... 29 common frames omitted
Caused by: java.lang.ClassNotFoundException: reactor.netty.http.client.HttpResponseDecoderSpec
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[na:1.8.0_192]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_192]
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:93) ~[gateway-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_192]
... 33 common frames omitted