Measuring response times (routing times) for each framework (middleware). Each framework has to have two features; routing and parsing path parameters.
Last update: 2018-03-15
OS: Darwin (version: 17.3.0, arch: x86_64)
CPU Cores: 8
- nickel (rust)
- japronto (python)
- iron (rust)
- router_cr (crystal)
- spider-gazelle (crystal)
- raze (crystal)
- actix (rust)
- fasthttprouter (go)
- amber (crystal)
- lucky (crystal)
- rocket (rust)
- kemal (crystal)
- clusterpolka (node)
- iris (go)
- gorilla_mux (go)
- echo (go)
- gin (go)
- vapor (swift)
- aspnetcore (csharp)
- perfect (swift)
- sanic (python)
- polka (node)
- kitura (swift)
- clusterexpress (node)
- plug (elixir)
- phoenix (elixir)
- akkahttp (scala)
- express (node)
- roda (ruby)
- rack-routing (ruby)
- criollo (objc)
- jester (nim)
- sinatra (ruby)
- tornado (python)
- rails (ruby)
- rust (nickel)
- python (japronto)
- crystal (router_cr)
- go (fasthttprouter)
- node (clusterpolka)
- swift (vapor)
- csharp (aspnetcore)
- elixir (plug)
- scala (akkahttp)
- ruby (roda)
- objc (criollo)
- nim (jester)
Language (Runtime) | Framework (Middleware) | Max [sec] | Min [sec] | Ave [sec] |
---|---|---|---|---|
ruby | rails | 183.065177 | 182.594138 | 182.784460 |
ruby | sinatra | 54.599611 | 54.235050 | 54.360554 |
ruby | roda | 19.539005 | 19.362119 | 19.428262 |
ruby | rack-routing | 24.912685 | 24.787798 | 24.857921 |
crystal | kemal | 5.239213 | 4.743690 | 4.857822 |
crystal | router_cr | 4.066965 | 3.929231 | 4.015780 |
crystal | raze | 4.274440 | 3.952904 | 4.073834 |
crystal | lucky | 4.407409 | 4.320964 | 4.377559 |
crystal | amber | 4.422603 | 4.302611 | 4.371277 |
crystal | spider-gazelle | 4.111804 | 3.961091 | 4.038874 |
go | echo | 5.706486 | 5.195444 | 5.458082 |
go | gorilla_mux | 5.688501 | 5.039164 | 5.442471 |
go | iris | 5.540159 | 5.014120 | 5.355044 |
go | fasthttprouter | 4.363762 | 4.202933 | 4.299051 |
go | gin | 6.022577 | 5.461489 | 5.817580 |
rust | actix | 4.239260 | 4.058780 | 4.103622 |
rust | iron | 3.980654 | 3.918856 | 3.944900 |
rust | nickel | 3.922851 | 3.836264 | 3.886912 |
rust | rocket | 4.922653 | 4.449431 | 4.709064 |
node | express | 15.727199 | 14.595424 | 14.948420 |
node | clusterexpress | 10.672635 | 8.388129 | 9.267956 |
node | polka | 8.788398 | 8.311747 | 8.478669 |
node | clusterpolka | 5.425973 | 5.079739 | 5.266841 |
elixir | plug | 10.217389 | 9.375499 | 9.780671 |
elixir | phoenix | 10.176699 | 9.456586 | 10.015052 |
swift | vapor | 6.805480 | 6.259005 | 6.561916 |
swift | perfect | 7.729230 | 7.527969 | 7.624530 |
swift | kitura | 9.018302 | 8.751379 | 8.857953 |
scala | akkahttp | 11.285840 | 9.558254 | 10.281555 |
csharp | aspnetcore | 7.064339 | 6.611566 | 6.810423 |
python | sanic | 8.930118 | 7.300009 | 8.129179 |
python | japronto | 4.255003 | 3.793259 | 3.926455 |
python | tornado | 99.925639 | 98.832093 | 99.228150 |
nim | jester | 35.604124 | 34.916220 | 35.241652 |
objc | criollo | 32.341102 | 31.504481 | 31.896720 |
- Ruby
- Crystal
- Go
- Rust
- node
- Elixir
- Swift
- Scala
- C#
- Python
- Nim
- Objective-C
See Development section when you want to add new languages or frameworks.
We want to know the response time (routing time), not a usability. So full-stack framework is at a disadvantage.
- Each server has no special logics.
- Each server's executable is named as
server_[Lauguage]_[Framework]
. (For example,server_ruby_sinatra
) - There are only 3 routes
- GET '/' return status code 200 with empty body
- GET '/user/:id' return status code 200 with the id
- POST '/user' return status code 200 with empty body
Required environment -> See Current target frameworks(middlewares)
Neph is a modern command line job processor that can be substitute for make
command.
To compile servers and benchmarker,
> neph
For each language,
> neph ruby
For each framework,
> neph rails
See neph.yaml
To compile servers and benchmarker,
> make
For each language,
> make ruby
For each framework,
> make rails
You can take a benchmark by
> bin/benchmarker
For each language
> bin/benchmarker ruby
For each framework
> bin/benchmarker rails
For comparison (Comparing rails, kemal and router.cr in this example)
> bin/benchmarker rails crystal
If you take it manually, you can run each server by
> bin/server_[Language]_[Framework]
and run client by
> time bin/client
You can set # of threads and # of the loops of the request(there are 3 requests in a loop) by
> time bin/client -t 16 -r 1000
In the above example, 16 threads requests 1000 * 3 times. So 48000 requests are sent in total.
Setup servers by using docker
is under WIP. Currently, crystal and ruby servers are supported. For example
docker-compose up rails
Then you can run your client by
time ./bin/client
- Give me PR when you want to add other web frameworks
- Give me PR when you can tuning each framework (under the rule)
/[language]/[framework]/[codes]
<- Project itselfbenchmarker/benchmarker.cr
<- Adding it as a target toREADME.md
<- Adding it as a target framework of the listMakefile
neph.yaml
(optional)
Anyway, you don't have to care about details since maintainer can fix them after merging it. The result will be updated by maintainer.
- Fork it (https://github.com/tbrand/which_is_the_fastest/fork)
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am 'Add some feature')
- Push to the branch (git push origin my-new-feature)
- Create a new Pull Request
- tbrand Taichiro Suzuki - creator, maintainer
- OvermindDL1 OvermindDL1 - maintainer
1AE9P6TUVik1rJGQhaSqGWRk1oAQ3DJnmo