sile/hb

A tiny HTTP benchmark tool written in Rust

hb

HTTP Benchmark Tool

hb's Issues

Performance issue when a response body of a GET request is large(> MiB)

Overview

hb run seems to consume most of execution time to decode a response body when a response body of a GET request is large.

The following line may be a root cause of this issue:

hb/src/run.rs

Line 166 in 10b2a8a

content: String::from_utf8(response.into_body()).ok(),

How to reproduce

Suppose that you have a frugalos cluster, run the following commands:

$ jo -a `for n in $(seq 0 1000); do jo method=PUT content=3145728 url=http://localhost:3000/v1/buckets/test/objects/$n; done` > /tmp/req1.json
$ jo -a `for n in $(seq 0 1000); do jo method=GET url=http://localhost:3000/v1/buckets/test/objects/$n; done` > /tmp/req2.json
$ hb run -i /tmp/req1.json | hb summary
$ hb run -i /tmp/req2.json | hb summary

hb run -i /tmp/req2.json | hb summary will take very long time.

Reference

The performance comparison is like below:

$ time ~/.cargo/bin/hb run -i ./req.json | hb summary
{
  "count": {
    "total": 1001,
    "ok": 1001,
    "error": 0
  },
  "status": {
    "200": 1000,
    "404": 1
  },
  "duration": 8.034492451,
  "rps": 124.58783253638033,
  "latency": {
    "min": 0.05501402,
    "median": 0.245011632,
    "mean": 0.2533950855684318,
    "max": 0.563494181,
    "var": 0.004940090512700124,
    "sd": 0.07028577745675241
  }
}
~/.cargo/bin/hb run -i ./req.json  29.36s user 2.95s system 48% cpu 1:06.85 total
hb summary  57.26s user 1.32s system 87% cpu 1:06.87 total

$ time /usr/local/bin/hb run -i ./req.json | hb summary
{
  "count": {
    "total": 1001,
    "ok": 1001,
    "error": 0
  },
  "status": {
    "200": 1000,
    "404": 1
  },
  "duration": 7.891263113,
  "rps": 126.8491476796612,
  "latency": {
    "min": 0.013134238,
    "median": 0.24080899,
    "mean": 0.24977976309790215,
    "max": 0.614390552,
    "var": 0.005919048378247439,
    "sd": 0.07693535194075243
  }
}
/usr/local/bin/hb run -i ./req.json  0.93s user 0.40s system 16% cpu 7.914 total
hb summary  0.00s user 0.01s system 0% cpu 7.913 total

~/.cargo/bin/hb was installed by cargo install hb and /usr/local/bin/hb was built by me. The diff is

$ git diff
diff --git a/Cargo.lock b/Cargo.lock
index e05ed35..23e7a14 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1,3 +1,5 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
 [[package]]
 name = "ansi_term"
 version = "0.11.0"
diff --git a/src/run.rs b/src/run.rs
index eb861fa..facff6d 100644
--- a/src/run.rs
+++ b/src/run.rs
@@ -163,7 +163,7 @@ impl Future for RunRequest {
             let response = Response {
                 status: response.status_code().as_u16(),
                 content_length: response.body().len() as u64,
-                content: String::from_utf8(response.into_body()).ok(),
+                content: None,
             };
             Ok(Async::Ready(response))
         } else {