I follow tutorial and install crank-agent and start it.
crank --config samples/hello/hello.benchmarks.yml --scenario hello --profile local
[06:35:46.635] Running session 'c1d5e040be52474c97d0c1ca42b0d634' with description ''
[06:35:46.690] Starting job 'application' ...
[06:35:46.961] Fetching job: http://localhost:5010/jobs/1
[06:36:39.408] Job has been selected by the server ...
[06:36:39.413] Job is now building ...
[06:42:08.029] Job is running...
[06:42:08.034] Starting job 'load' ...
[06:42:08.040] Fetching job: http://localhost:5010/jobs/2
[06:42:10.069] Job has been selected by the server ...
[06:42:10.071] Job is now building ...
[06:42:23.175] Job is running...
[06:44:41.675] Stopping job 'load' ...
[06:44:42.760] Deleting job 'load' ...
[06:44:42.804] Stopping job 'application' ...
[06:44:43.828] Deleting job 'application' ...
application
-------
## Host Process:
CPU Usage (%): 5
Raw CPU Usage (%): 38.53
Working Set (MB): 52
Build Time (ms): 14,211
Start Time (ms): 4,408
Published Size (KB): 86,819
load
-------
## Host Process:
CPU Usage (%): 3
Raw CPU Usage (%): 20.17
Working Set (MB): 33
Build Time (ms): 5,223
Start Time (ms): 41
Published Size (KB): 68,190
## Benchmark:
First Request (ms): 306
Requests: 512
Bad responses: 512
Mean latency (us): 10,096,008
Max latency (us): 10,609,015
Requests/sec: 17
Requests/sec (max): 3,350
I notice that there 512 failed requests, and attempt to run again with same command.
I have only partial results
[07:14:33.811] Running session 'd56a13d2506345abbf5ec7a4d38ae2df' with description ''
[07:14:33.860] Starting job 'application' ...
[07:14:33.903] Fetching job: http://localhost:5010/jobs/7
[07:14:34.926] Job has been selected by the server ...
[07:14:34.930] Job is now building ...
[07:14:50.047] Job is running...
[07:14:50.052] Starting job 'load' ...
[07:14:50.055] Fetching job: http://localhost:5010/jobs/8
[07:14:52.071] Job has been selected by the server ...
[07:14:52.073] Job is now building ...
[07:15:05.216] Job is running...
[07:14:34.640] Processing job 'application' (7) in state New
[07:14:34.641] Acquiring Job 'application' (7)
[07:14:34.641] Checking requirements...
[07:14:34.642] New -> Initializing
[07:14:35.645] Processing job 'application' (7) in state Waiting
[07:14:35.645] Starting job 'application' (7)
[07:14:35.645] Waiting -> Starting
[07:14:35.646] Created temp directory 'C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr'
[07:14:35.647] Cloning https://github.com/dotnet/crank with branch 'master'
[07:14:35.755] [STDERR] Cloning into 'crank'...
[07:14:38.506] Benchmarked Application in C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello
[07:14:38.507] Downloading build tools
[07:14:38.507] Installing dotnet runtimes and sdk
[07:14:38.510] Detected target framework: 'netcoreapp3.1'
[07:14:38.511] Patching project file with Framework References
[07:14:41.288] Runtime: 3.1.6 (Current)
[07:14:41.288] SDK: 3.1.302 (Current)
[07:14:41.289] ASP.NET: 3.1.6 (Current)
[07:14:41.289] Patching existing global.json file
[07:14:41.292] Desktop: 3.1.6 (Current)
[07:14:41.293] Publishing application in C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published with:
publish hello.csproj -c Release -o C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published /p:MicrosoftNETCoreAppPackageVersion=3.1.6 /p:MicrosoftWindowsDesktopAppPackageVersion=3.1.6 /p:MicrosoftAspNetCoreAppPackageVersion=3.1.6 /p:BenchmarksNETStandardImplicitPackageVersion=3.1.6 /p:BenchmarksNETCoreAppImplicitPackageVersion=3.1.6 /p:BenchmarksRuntimeFrameworkVersion=3.1.6 /p:BenchmarksTargetFramework=netcoreapp3.1 /p:BenchmarksAspNetCoreVersion=3.1.6 /p:MicrosoftAspNetCoreAllPackageVersion=3.1.6 /p:NETCoreAppMaximumVersion=99.9 /p:MicrosoftNETCoreApp31PackageVersion=3.1.6 /p:MicrosoftNETPlatformLibrary=Microsoft.NETCore.App /p:RestoreNoCache=true --framework netcoreapp3.1 --self-contained -r win-x64
[07:14:47.451] Application published successfully in 6156.6666 ms
[07:14:47.452] Patching hello.runtimeconfig.json
[07:14:47.454] Application is self-contained, skipping runtimeconfig.json
[07:14:47.455] Published size: 86819
[07:14:47.455] Invoking executable: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published\hello.exe
[07:14:47.456] Arguments:
[07:14:47.456] Working directory: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published
[07:14:47.499] Process started: 29640
[07:14:47.499] Starting measurement
[07:14:48.739] Processing job 'application' (7) in state Starting
[07:14:49.564] AspNetCore location: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published\Microsoft.AspNetCore.Hosting.Abstractions.dll
[07:14:49.564] AspNetCore version: 3.1.6
[07:14:49.564] NETCoreApp location: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published\System.Private.CoreLib.dll
[07:14:49.564] NETCoreApp version: 3.1.5-servicing.20269.1+018cfd06dceb19b6eb1e9217a500fb1071946fcd
[07:14:49.689] info: Microsoft.Hosting.Lifetime[0]
[07:14:49.689] Now listening on: http://[::]:5000
[07:14:49.703] info: Microsoft.Hosting.Lifetime[0]
[07:14:49.703] Application started. Press Ctrl+C to shut down.
[07:14:49.704] Running job 'application' (7)
[07:14:49.704] info: Microsoft.Hosting.Lifetime[0]
[07:14:49.704] Hosting environment: Production
[07:14:49.705] info: Microsoft.Hosting.Lifetime[0]
[07:14:49.705] Content root path: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr\crank\samples\hello\published
[07:14:49.750] Processing job 'application' (7) in state Running
[07:14:50.755] Adding job 'load' (8) to group
[07:14:50.755] Processing job 'application' (7) in state Running
[07:14:51.765] Processing job 'load' (8) in state New
[07:14:51.765] Acquiring Job 'load' (8)
[07:14:51.765] Checking requirements...
[07:14:51.766] New -> Initializing
[07:14:52.770] Processing job 'application' (7) in state Running
[07:14:53.779] Processing job 'load' (8) in state Waiting
[07:14:53.779] Starting job 'load' (8)
[07:14:53.780] Waiting -> Starting
[07:14:53.780] Created temp directory 'C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua'
[07:14:53.781] Cloning https://github.com/dotnet/crank.git with branch 'master'
[07:14:53.885] [STDERR] Cloning into 'crank'...
[07:14:56.843] Benchmarked Application in C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua\crank\src\Microsoft.Crank.Jobs.Bombardier
[07:14:56.843] Downloading build tools
[07:14:57.156] Installing dotnet runtimes and sdk
[07:14:57.158] Detected target framework: 'netcoreapp3.1'
[07:14:57.158] Patching project file with Framework References
[07:15:00.233] Runtime: 3.1.6 (Current)
[07:15:00.234] SDK: 3.1.302 (Current)
[07:15:00.234] ASP.NET: 3.1.6 (Current)
[07:15:00.234] Patching existing global.json file
[07:15:00.238] Desktop: 3.1.6 (Current)
[07:15:00.239] Publishing application in C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua\crank\src\Microsoft.Crank.Jobs.Bombardier\published with:
publish Microsoft.Crank.Jobs.Bombardier.csproj -c Release -o C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua\crank\src\Microsoft.Crank.Jobs.Bombardier\published /p:MicrosoftNETCoreAppPackageVersion=3.1.6 /p:MicrosoftWindowsDesktopAppPackageVersion=3.1.6 /p:MicrosoftAspNetCoreAppPackageVersion=3.1.6 /p:BenchmarksNETStandardImplicitPackageVersion=3.1.6 /p:BenchmarksNETCoreAppImplicitPackageVersion=3.1.6 /p:BenchmarksRuntimeFrameworkVersion=3.1.6 /p:BenchmarksTargetFramework=netcoreapp3.1 /p:BenchmarksAspNetCoreVersion=3.1.6 /p:MicrosoftAspNetCoreAllPackageVersion=3.1.6 /p:NETCoreAppMaximumVersion=99.9 /p:MicrosoftNETCoreApp31PackageVersion=3.1.6 /p:MicrosoftNETPlatformLibrary=Microsoft.NETCore.App /p:RestoreNoCache=true --framework netcoreapp3.1 --self-contained -r win-x64
[07:15:05.081] Application published successfully in 4841.0721 ms
[07:15:05.082] Patching Microsoft.Crank.Jobs.Bombardier.runtimeconfig.json
[07:15:05.083] Application is self-contained, skipping runtimeconfig.json
[07:15:05.084] Published size: 68190
[07:15:05.084] Invoking executable: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua\crank\src\Microsoft.Crank.Jobs.Bombardier\published\Microsoft.Crank.Jobs.Bombardier.exe
[07:15:05.085] Arguments: -c 256 -w 0 -d 15 -n 0 --insecure -l --fasthttp http://localhost:5000/
[07:15:05.085] Working directory: C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua\crank\src\Microsoft.Crank.Jobs.Bombardier\published
[07:15:05.120] Running job 'load' (8)
[07:15:05.120] Process started: 24904
[07:15:05.120] Starting measurement
[07:15:06.871] Processing job 'application' (7) in state Running
[07:15:07.881] Processing job 'load' (8) in state Running
[07:15:08.890] Processing job 'application' (7) in state Running
[07:15:08.943] Bombardier Client
[07:15:08.943] args: -c 256 -w 0 -d 15 -n 0 --insecure -l --fasthttp http://localhost:5000/
[07:15:09.269] Measuring first request ... 306 ms
[07:15:09.279] Downloading bombardier from https://github.com/codesenberg/bombardier/releases/download/v1.2.4/bombardier-windows-amd64.exe to bombardier-windows-amd64.exe
[07:15:09.906] Processing job 'load' (8) in state Running
[07:15:10.917] Processing job 'application' (7) in state Running
......
[07:15:48.231] Processing job 'load' (8) in state Running
[07:15:49.246] Processing job 'application' (7) in state Running
[07:15:49.731] > bombardier -d 15s -c 256 --insecure -l --fasthttp http://localhost:5000/ --print r --format json
[07:15:50.261] Processing job 'load' (8) in state Running
[07:15:50.375] [STDERR] ##ChildProcessId:27176
[07:15:51.262] Processing job 'application' (7) in state Running
[07:15:52.263] Processing job 'load' (8) in state Running
[07:15:53.269] Processing job 'application' (7) in state Running
[07:15:54.278] Processing job 'load' (8) in state Running
[07:15:55.285] Processing job 'application' (7) in state Running
[07:15:56.285] Processing job 'load' (8) in state Running
[07:15:57.294] Processing job 'application' (7) in state Running
[07:15:58.299] Processing job 'load' (8) in state Running
[07:15:59.314] Processing job 'application' (7) in state Running
[07:15:59.576] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'application' (7).
[07:15:59.576] Running -> Stopping
[07:16:00.326] Processing job 'load' (8) in state Running
[07:16:00.528] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:00.529] Running -> Stopping
[07:16:00.590] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'application' (7).
[07:16:01.327] Processing job 'application' (7) in state Stopping
[07:16:01.327] Stopping job 'application' (7)
[07:16:01.328] Stopping measurement event pipes for job 'application' (7)
[07:16:01.544] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:02.559] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:03.580] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:04.592] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:05.606] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:06.621] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:07.626] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:08.636] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:09.653] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:10.669] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:10.994] {"spec":{"numberOfConnections":256,"testType":"timed","testDurationSeconds":15,"method":"GET","url":"http://localhost:5000/","body":"","stream":false,"timeoutSeconds":2,"client":"fasthttp"},"result":{"bytesRead":0,"bytesWritten":158720,"timeTakenSeconds":20.596004,"req1xx":0,"req2xx":0,"req3xx":0,"req4xx":0,"req5xx":0,"others":512,"errors":[{"description":"the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection","count":512}],"latency":{"mean":1.0088474890625e+07,"stddev":65479.310496470105,"max":1.0576999e+07,"percentiles":{"50":10066038,"75":10121002,"90":10144001,"95":10148002,"99":10568014}},"rps":{"mean":17.368926103888505,"stddev":195.21480631463487,"max":3375.717339934736,"percentiles":{"50":0.000000,"75":0.000000,"90":0.000000,"95":0.000000,"99":45.608400}}}}
[07:16:11.684] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:16:11.684] Process has exited (0)
[07:16:11.685] Stopping -> Stopped
....
[07:17:26.358] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:26.359] Process has exited (0)
[07:17:26.359] Stopped -> Stopped
[07:17:27.372] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:27.373] Process has exited (0)
[07:17:27.373] Stopped -> Stopped
[07:17:28.374] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:28.375] Process has exited (0)
[07:17:28.375] Stopped -> Stopped
[07:17:29.388] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:29.389] Process has exited (0)
[07:17:29.389] Stopped -> Stopped
[07:17:30.390] Process has exited (0)
[07:17:30.390] Stopped -> Stopped
[07:17:31.316] Forcing process to stop ...
[07:17:31.322] Waiting for process 29640 to stop ...
[07:17:31.399] Process has exited (0)
[07:17:31.399] Stopped -> Stopped
[07:17:32.327] Process has stopped
[07:17:32.327] Process stopped (Stopped)
[07:17:32.410] Process has exited (0)
[07:17:32.410] Stopped -> Stopped
[07:17:33.339] Processing job 'load' (8) in state Stopped
[07:17:33.339] Job 'load' (8) has stopped, waiting for the driver to delete it
[07:17:33.422] Process has exited (0)
[07:17:33.422] Stopped -> Stopped
[07:17:34.345] Processing job 'application' (7) in state Stopped
[07:17:34.345] Job 'application' (7) has stopped, waiting for the driver to delete it
[07:17:34.346] Driver didn't communicate for 00:00:10. Halting job.
[07:17:34.346] Stopped -> Deleting
[07:17:34.435] Process has exited (0)
[07:17:34.435] Stopped -> Stopped
[07:17:35.355] Processing job 'load' (8) in state Stopped
[07:17:35.355] Job 'load' (8) has stopped, waiting for the driver to delete it
[07:17:35.445] Process has exited (0)
[07:17:35.445] Stopped -> Stopped
[07:17:36.364] Processing job 'application' (7) in state Deleting
[07:17:36.365] Deleting job 'application' (7)
[07:17:36.365] Deleting directory 'C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\uutkasr1.mmr'
[07:17:36.459] Process has exited (0)
[07:17:36.459] Stopped -> Stopped
[07:17:36.961] SUCCESS
[07:17:36.962] Deleting -> Deleted
[07:17:37.462] Process has exited (0)
[07:17:37.462] Stopped -> Stopped
[07:17:37.968] Processing job 'load' (8) in state Stopped
[07:17:37.968] Job 'load' (8) has stopped, waiting for the driver to delete it
[07:17:38.476] Process has exited (0)
[07:17:38.476] Stopped -> Stopped
[07:17:38.969] Processing job 'application' (7) in state Deleted
[07:17:39.489] Process has exited (0)
[07:17:39.490] Stopped -> Stopped
[07:17:39.981] Processing job 'load' (8) in state Stopped
[07:17:39.982] Job 'load' (8) has stopped, waiting for the driver to delete it
[07:17:40.499] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:40.499] Process has exited (0)
[07:17:40.500] Stopped -> Stopped
[07:17:40.983] Processing job 'application' (7) in state Deleted
[07:17:41.511] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:41.512] Process has exited (0)
[07:17:41.512] Stopped -> Stopped
[07:17:41.995] Processing job 'load' (8) in state Stopped
[07:17:41.996] Job 'load' (8) has stopped, waiting for the driver to delete it
[07:17:41.996] Driver didn't communicate for 00:00:10. Halting job.
[07:17:41.996] Stopped -> Deleting
[07:17:42.527] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:42.528] Process has exited (0)
[07:17:43.011] Processing job 'application' (7) in state Deleted
[07:17:43.544] [Heartbeat] Driver didn't communicate for 00:00:10. Halting job 'load' (8).
[07:17:43.544] Process has exited (0)
[07:17:44.026] Processing job 'load' (8) in state Deleting
[07:17:44.026] Deleting job 'load' (8)
[07:17:44.026] Stopping measurement event pipes for job 'load' (8)
[07:17:44.026] Process stopped (Deleting)
[07:17:44.027] Deleting directory 'C:\Users\kant\AppData\Local\Temp\benchmarks-agent\benchmarks-server-24280\1hfikjk4.jua'
[07:17:44.486] SUCCESS
[07:17:44.486] Deleting -> Deleted
[07:17:45.497] All jobs in group are finished