kffl/gocannon

:boom: Performance-focused HTTP load testing tool written in Go

gocannon's Issues

Add support for other HTTP request methods (POST, PUT, DELETE)

As of version 0.0.1, Gocannon only supports generating HTTP GET requests. A feature allowing for specifying the desired request method of the load test (i.e. --method=POST, which would default to GET when not provided) should be added.

User-defined HTTP request body

Related to #1

Once HTTP methods other than GET (POST, PUT, DELETE) are implemented, it would be great if the user could specify the contents of the request body.

Documentation improvements

The set of functionalities offered by Gocannon has expanded quite considerably in the last couple of releases. There are some areas of the documentation that could use a bit of an improvement:

• Add a short guide on writing custom Gocannon plugins
• Describe each config flag/arg in more detail
• Add information about the available docker container images
• Document the JSON/YAML output schema for both request log and histogram modes
• Add info about using Gocannon as a library/dependency for programmatic load tests

Request log report latency formatting improvements

The latency format in the request log report could use come improvements:

|--REQS--|    |------------------------LATENCY-------------------------|
     Count           AVG         P50         P75         P90         P99
       428   53.692646ms 34.784632ms 38.176138ms 114.18323ms 173.735751ms
       788   32.170186ms 32.059117ms 34.617489ms 39.436144ms 41.817704ms
       859   29.082034ms 28.592396ms 33.484341ms  34.90701ms 35.561211ms
       977     25.5611ms 25.658544ms 28.236734ms 30.303005ms  31.82007ms

In order to improve readability, latency results within the same order of magnitude should have constant length so that the decimal point is always in the same column across all rows.

Additionally, if no responses were recorded within a given interval, -1ns value is shown, which should be replaced with something more appropriate (i.e -).

Include documentation and license in archives with release binaries

As of now, binaries that are built and published upon each new release (#12) don't contain the documentation (README.md) and LICENSE files. Appropriate tweaks should be made to go-release-action action config so as to include these files.

--header flag is not recognized

Since 7618e5b, the --header flag is not being recognized:

> gocannon --header="Content-Type: application/json" http://target:8080/url
error: unknown long flag '--header'

ouput the statistics when Ctrl+C pressed

"github.com/kffl/gocannon" is a program, not an importable package on build when using as a library

I'm getting the following error when I try to use gocannon as a library in a program:

> go build
main.go:9:2: import "github.com/kffl/gocannon" is a program, not an importable package

Code:

package main

import (
	"fmt"
	"os"
	"runtime"
	"time"

	gocannon "github.com/kffl/gocannon"
	"github.com/kffl/gocannon/common"
)

func main() {
	duration := time.Second * 1
	connections := 50
	cpus := runtime.NumCPU()
	timeout := time.Millisecond * 200
	mode := "reqlog"
	outputFile := ""
	interval := time.Millisecond * 250
	preallocate := 1000
	method := "GET"
	body := common.RawRequestBody{}
	header := common.RequestHeaders{}
	trustAll := false
	format := "default"
	plugin := ""
	target := "http://localhost:8080/v1/dict/roles"

	cfg := common.Config{
		Duration:    &duration,
		Connections: &connections,
		CPUs:        &cpus,
		Timeout:     &timeout,
		Mode:        &mode,
		OutputFile:  &outputFile,
		Interval:    &interval,
		Preallocate: &preallocate,
		Method:      &method,
		Body:        &body,
		Headers:     &header,
		TrustAll:    &trustAll,
		Format:      &format,
		Plugin:      &plugin,
		Target:      &target,
	}
	g, err := gocannon.NewGocannon(cfg)

	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	g.Run()
}

Add support for HTTPS?

It looks like trying to use a HTTPS url returns ErrUnsupportedProtocol at this time.

Automatic building and publishing of release binaries for multiple platforms

A GitHub action should be configured so as to automatically build and publish Linux, Windows and macOS x64 binaries upon each release.

CI action sometimes fails during integration/system test

The CI workflow sometimes fails when performing TestGocannon:

 === RUN   TestGocannon
    assertion_compare.go:342: 
        	Error Trace:	integration_test.go:78
        	Error:      	"2.1263238042593002" is not less than or equal to "2"
        	Test:       	TestGocannon
        	Messages:   	[average latencies calculated by reqlog and hist should be within the error margin]

During this test, logic identical to runGocannon() is performed by load testing a sample HTTP server (target_test.go). The only difference when compared to an actual run is that performance metrics are collected with both the request log and latency histogram (as well as atomic counters) so as to ensure that all methods obtain equal results (within the specified error margin when calculating the average).

When running the test on GitHub Actions workers, the difference between average request latency obtained via the request log and latency histogram statistic collectors sometimes exceeds the given threshold of 2 nanoseconds. Since the potential discrepancy between the two methods is proportional to the request latencies obtained during the test, the problem is likely caused by GitHub Action workers having less resources than my local setup, on which I initially developed the tests (I'm getting averages of about 0.5ms on local vs ~10ms in CI). In order to fix this issue, the test should be modified so as to calculate the maximum threshold as a percentage of the obtained value (i.e 0.01%), not a fixed 2ns value.

A flag to specify logical CPUs count

As of now the user can specify the maximum number of logical CPUs, on which Gocannon threads can be executed simultaneously by setting the GOMAXPROCS environment variable, which is not the most convenient way of interacting with the program. A separate flag (--cpus) could be added so as to allow the user to specify that value.

Custom HTTP headers

As of version 0.0.1, there is no way for the user to attach custom HTTP headers to each request sent by Gocannon during the load test. This feature should enable the user to specify HTTP headers (i.e. via --headers="Content-Type: application/json"). Such flag should be repeatable so as to allow for setting multiple headers.

Output load test report as JSON or YAML

As of now, Gocannon prints the load test summary/report to std out using a rather verbose/tabular format, which is difficult to parse. In order to make it more convenient to include Gocannon in CI/CD pipelines (i.e running a load test, results of which are piped to jq), alternative report output formats (JSON and YAML) should be added.

Using Gocannon as a library

As of now, Gocannon can only be used as a standalone program (with optional support for custom plugins). After making some changes to the codebase (exposing exported methods and tying the config struct to a specific instance of Gocannon), it would be possible to use Gocannon as an external library called from other Go code (i.e. for the purpose of programmatically performing a load test).

Check if target host is reachable before the load test

As of now, load test target validation includes some checks (URL parsing being successful, protocol being supported and the port being present), but there no additional steps undertaken in order to determine if the target host is actually reachable (i.e. does its domain name DNS query resolve properly?; can a TCP connection be established?). Consequently, if Gocannon is unable to connect to the host, the user will find out about that fact only after the load test ends (with none of the requests being completed successfully).

This feature could be implemented by TCP dialing the target host prior to the load test start and returning a descriptive error if such operation is unsuccessful.

Support user-defined scripts via Go plugins

Since as of now, Gocannon only supports performing load tests, in which every HTTP request is identical, it might be useful to let the user programmatically alter the program's behavior during the load test so as to allow for executing more complex load testing scenarios.

While I initially wanted to avoid embedding a compiler/runtime like LuaJIT or V8 in Gocannon due to performance considerations, similar results can be achieved by implementing support for support for Go plugins, that would make it possible to inject custom logic at program startup, teardown as well as before and after each request is sent (in an aspect-oriented programming fashion). Development of a user-defined plugin would rely on creating a new package exposing a pre-defined interface and compiling it to a shared object, which would then be accessed and executed by Gocannon at runtime (i.e. with the path to the .so file provided via --plugin=path/to/plugin.so).