tsenart/vegeta

HTTP load testing tool and library. It's over 9000!

vegeta's Issues

report sometimes produces no output

Sometimes when I run report, it produces no output at all. However, if I run the exact same command a few more times, eventually the output is produced as expected. I not familiar with Go. Any idea what could be happening here?

Histogram reporter

Viewing latencies distributions in a histogram is useful for certain types of data analysis. This has been originally proposed with #78.

Feature request: fixed size plots

A plot could be constrained to a fixed size, and data points scaled to fit.

Down/upsampling may be required. (I've heard that the maximum number of human-understandable data points in a single graph is 400.)

Feature request: more robust plot output

Plotinum appears to have multiple runtime dependencies. It would be cool to reduce complexity there somehow, so I can confidently generate plots on remote machines or whatever.

Failed to locate a font file NimbusRomNo9L-Regu.ttf

./vegeta -duration=5s -ordering="random" -rate=50 -reporter="plot:timings"
2013/08/30 17:08:26 Vegeta is attacking 1 targets in random order for 5s...
2013/08/30 17:08:32 Done!
2013/08/30 17:08:32 Writing report to 'stdout'...
2013/08/30 17:08:32 Failed to report: Failed to locate a font file NimbusRomNo9L-Regu.ttf for font name Times-Roman

Added file to osx fonts and to current folder path, but it still doesn't find it.

Output requests that are over a configurable threshold

It is nice to get all errors logged in vegeta report.

For many use cases a requests that takes longer than a certain threshold is almost as bad as a call that throws a server error or times out.

Therefore it would be useful to be able to pass a threshold to vegeta like:
vegeta attack -threshold=2s ... | vegeta report

All requests that take longer than that request should then be listed like this:

Error Set:
<all URLs with some HTTP error>
Above Threshold Set (2s):
<all URLs above 2s threshold>

One could also add a horizontal line for the threshold in the output of vegeta report -reporter=plot.

Is this a good idea? And is this doable?
I don't know go much but I am happy to help on this.

report saying network is unreachable + connection refused

Hi,

first of all great tool I am loving it.

I am running a rate 20 attack on a heroku app (mine) on a test plan of around 2K urls for about 1H.

when the reports completes I am seeing lots of:

 dial tcp: read udp 172.25.64.30:53: connection refused
URL dial tcp: dial udp 172.25.64.30:53: network is unreachable
 dial tcp: dial udp 172.25.64.30:53: network is unreachable
Get URL dial tcp: i/o timeout

and here's the report output:

Requests        [total]                         71999
Duration        [total, attack, wait]           1h0m0.201463185s, 59m59.952162509s, 249.300676ms
Latencies       [mean, 50, 95, 99, max]         695.300964ms, 256.873692ms, 1.242270337s, 30.775127016s, 30.775127016s
Bytes In        [total, mean]                   4438273450, 61643.54
Bytes Out       [total, mean]                   0, 0.00
Success         [ratio]                         96.68%
Status Codes    [code:count]                    200:69606  505:717  0:1676
Error Set:

I am running vegeta on a 2.6Ghz i7 macbook pro, OSX 10.9.5 do you reckon those errors are caused by me reaching the OS NIC limits?

Thanks

http proxy with NTLM authentication support

Just dropping an idea..
A feature that I use a lot with curl-loader: http web proxy address with NTLM authentication support.

SVG not generated

Hi

I'm using the linux64 binary with : ./vegeta -duration=60s -ordering="sequential" -output="out" -rate=20 -reporter="plot:timings" -targets="targets.txt"

However, no SVG is generated, the "out" file is 0 bytes. The text report works fine.

Am I doing something wrong ?

laurent

surpport http 1.1 without keep-alive?

the 9000 concurrents mean keep-alive?

Unclear documentation of vegeta.NewAttacker

// NewAttacker returns a pointer to a new Attacker
//
// redirects is the max amount of redirects the attacker will follow.
// timeout is the client side timeout for each request.
// addr is the local IP address used for each request.
// If nil, a local IP address is automatically chosen.
func NewAttacker(redirects int, timeout time.Duration, laddr net.IPAddr) *Attacker {

• the third parameter is called laddr
• it's not possible to pass nil

Support HTTPS

Hi.

Your README gives HTTPS support as a TODO, but after a quick glance at the code, I can't see why it wouldn't already work? I'm happy to contribute, since it's a very useful-looking tool but all my sites use HTTPS only. What sort of changes need to be made to add HTTPS support?

Ability to send a unique body for each target

Based on the documentation I don't see a way to presently do this. For example I would like to

GET /step/1/
POST key=val /step/1/
GET /step/2/
POST foo=bar /step/2/
...

It seems like this flow would be pretty common. The way I understand the documentation, any body I set will be sent to all targets. Am I misunderstanding that?

Output is garbled

Feature: signal vegeta to increase/decrease attack rate

Would be great if vegeta could receive signals (TTIN/TTOU)[http://unicorn.bogomips.org/SIGNALS.html] to increase rate during the attack.

This could be used to simulate traffic waves.

Ability to save cookies

When using a series of targets, if a request sends a cookie in it's response, it would be super handy to have the option to save that cookie and use it in successive requests. I understand there are potential memory concerns there but in similar projects, it is a super handy option when simulating things like e-commerce transactions.

Setting high duration's attempts to allocate a ton of memory

I'm using Vegeta that are being considered as alternatives to current production servers, and I noticed that inside a 2gb VM, if you set an overly high duration, in this case 500h (typo'd the m to an h), Vegeta takes some time to think and then usually ends up running out of memory:

goroutine 1 [running]:
[fp=0x7f40498e4b20] runtime.throw(0x9e1885)
    /Users/tomas/Code/gosrc/src/pkg/runtime/panic.c:473 +0x67
[fp=0x7f40498e4b78] runtime.mallocgc(0x10c388d058, 0x100000000, 0x1)
    /Users/tomas/Code/gosrc/src/pkg/runtime/zmalloc_linux_amd64.c:63 +0x344
[fp=0x7f40498e4b98] runtime.mal(0x10c388d058)
    /Users/tomas/Code/gosrc/src/pkg/runtime/zmalloc_linux_amd64.c:614 +0x3f
[fp=0x7f40498e4be8] runtime.makechan_c(0x686de0, 0x218711a00)
    /Users/tomas/Code/gosrc/src/pkg/runtime/chan.c:114 +0xba
[fp=0x7f40498e4c00] runtime.makechan(0x686de0, 0x218711a00, 0xc200122000)
    /Users/tomas/Code/gosrc/src/pkg/runtime/chan.c:141 +0x2f
[fp=0x7f40498e4c40] github.com/tsenart/vegeta/lib.Attack(0xc2000df880, 0x3, 0x4, 0x1388, 0x6651728988000, ...)
    /Users/tomas/Code/go/src/github.com/tsenart/vegeta/lib/attack.go:16 +0x138
[fp=0x7f40498e4f90] main.main()
    /Users/tomas/Code/go/src/github.com/tsenart/vegeta/main.go:80 +0x824
[fp=0x7f40498e4fb8] runtime.main()
    /Users/tomas/Code/gosrc/src/pkg/runtime/proc.c:182 +0x92
[fp=0x7f40498e4fc0] runtime.goexit()
    /Users/tomas/Code/gosrc/src/pkg/runtime/proc.c:1223
runtime: out of memory: cannot allocate 72000012288-byte block (72002109440 in use)
fatal error: out of memory

This isn't a big problem (I would imagine no one really wants to run vegeta for that long), but it would be a good idea to limit or warn users about this shortcoming.

Maybe mishandling error cases?

https://github.com/tsenart/vegeta/blob/master/lib/attack.go#L62

if body, err := ioutil.ReadAll(r.Body); err != nil && result.code < 200 || result.code >= 300 {
    result.err = errors.New(string(body))
}

This will set result.err if err == nil and result.code == e.g. 301 (I think a false positive) and will not set result.err if err != nil and result.code < 300 (I think a false negative).

Maybe I misinterpret your goals here though.

targets file support http basic auth

Is this possible?

Follow 0 redirects registers each request as error for a redirect

I work in ad tech and we have a server that primarily does redirects to external resources and I do not want their services included in my tests. Since I do not want to follow redirects and most end points give a 302 or a 307, vegeta declares it as an error here. This seems to be good default behavior but it'd be nice if I could supply an option to so that it would not follow and not register as an error.

Do you like any of these options?

• -ignore-redirects Sets -redirects=0 and ensures status codes in the 300s are not counted as errors.
• Don't register 300s as errors when -redirects=0
• Change the default behavior so that 300s don't automatically count as errors.

multiple sources

This tool is awesome and it has everything I need for the testing I do. I just need to be able to generate stateful traffic from multiple source addresses. Would it be possible to add a feature that will you to bind a /24 or arbitrary number of IP addresses and then split the traffic among these IP's on the same host? There is another project called curl-loader which is the only tool that I know that has this capability. It would be great if it is possible to add this feature to your tool. I see that you have documented a method to execute the tool on multiple hosts but I need to be able to use a /24 or more of address space.

thanks!

Ability to use output of previous attack for a new attack

Assuming I run an attack like this:

vegeta attack -duration=10s -rate=2 -targets=targets.txt > results.vr

It would be great if I could given results.vr to somebody and tell them to run it again like:

vegeta attack -playback=results.vr

This would then run the attack exactly the way that it was run the first time.

This assumes that all parameters used for the 1st attack could be persisted within results.vr somehow, including all URLs used. This would also be helpful in order to display the attack parameters in the different report forms, in order to distinguish different attacks from each other.

Doubt in rate and workers

One of the doubts I have is the workers flag. For example, I have a rate of 1000 req/s and 1 worker... this mean I'll have one thread sending the 1000 requests?
What about 10 workers? Means that I have 10 threads sending 1000 requests in total right? Not each one.

Is possible to configure Vegeta to have 10 workers sending 1000 req/s each? Not in total.

Option to ignore SSL errors

I have for example cloudflare -> elastic load balancer -> instances and I want to attack a specific endpoint, for example by giving the DNS name of the elastic load balancer. Unfortunately, this doesn't work because it has a different DNS name than the SSL certificate it is using.

can't load package: package github.com/tsenart/vegeta: import "github.com/tsenart/vegeta": cannot find package

is my go too old maybe? 1.0.3

duration 420m streams messages on STDOUT

using a duration higher then 420 starts displaying on STDOUT a stream of:

goroutine 114338 [select]:
^Created by github.com/tsenart/vegeta/lib.(*Attacker).Attack
        /Users/tomas/Code/go/src/github.com/tsenart/vegeta/lib/attack.go:154 +0x382

is it just a warning? I did not go trough running the app. Having 240 doesn't cause that stream.

go get github.com/tsenart/vegeta

first I use apt-get install golang and find it installs in the usr/bin/go, then I just set GOPATH variable and then exe "go get github.com/tsenart/vegeta " , then it report error

github.com/tsenart/vegeta/lib

../../go/src/github.com/tsenart/vegeta/lib/attack.go:46: unknown net.Dialer field 'KeepAlive' in struct literal
../../go/src/github.com/tsenart/vegeta/lib/attack.go:53: unknown http.Transport field 'TLSHandshakeTimeout' in struct literal

who can help me to resolve the problem.

404s not reported in the HTML5 graph

I have created a targets file with a number of "bad" URLs that return 404s. The 404s are noted in the JSON object 'status_codes":{"200":95,"404":5}', but not in the HTML5 graph, which shows responses as 'OK'. I wondered if this was a defect or purpose design?

Report plot with status code detail

Do you think it is possible to plot a report where each status code is drawed in its own line?
For example let's say that I execute 200 request, 50 requests have HTTP 200, 150 have HTTP 500. I think it should be useful to have different lines in the output.

regards
Davide Icardi

Plotting a image

Hey,

I'd like to suggest to let vegeta reporter also support plotting into a image. I would need it because I've quite a bunch of result files and would like to compare the results with multiple images.

cheers! 🏮

go version go1.2.1 linux/amd64 causes the following errors

Hi,

ubuntu 14.04 uses go version go1.2.1 linux/amd64 causes the following errors:

ubuntu@ip:~/workspace$ go get github.com/tsenart/vegeta
# github.com/tsenart/vegeta/lib
src/github.com/tsenart/vegeta/lib/attack.go:45: unknown net.Dialer field 'KeepAlive' in struct literal
src/github.com/tsenart/vegeta/lib/attack.go:54: unknown http.Transport field 'TLSHandshakeTimeout' in struct literal
src/github.com/tsenart/vegeta/lib/attack.go:113: a.dialer.KeepAlive undefined (type *net.Dialer has no field or method KeepAlive)

ubuntu@ip:~/workspace$ go install github.com/tsenart/vegeta
# github.com/tsenart/vegeta/lib
src/github.com/tsenart/vegeta/lib/attack.go:45: unknown net.Dialer field 'KeepAlive' in struct literal
src/github.com/tsenart/vegeta/lib/attack.go:54: unknown http.Transport field 'TLSHandshakeTimeout' in struct literal
src/github.com/tsenart/vegeta/lib/attack.go:113: a.dialer.KeepAlive undefined (type *net.Dialer has no field or method KeepAlive)

what's the recommended go version to install vegeta?

compressed graph output

gets pretty massive when you crank the rate up, but otherwise very cool

invalid URLs are reported as 505 HTTP Version Not Supported

I had a few malformed URLs containing markup left from a crawling an HTML page. This is definitely user mistake but vegeta was reporting those as 505 (505 HTTP Version Not Supported
The server does not support the HTTP protocol version used in the request.)

What do you think about flag and ignore invalid URLs?

Example URL causing problem:

GET valid_url" class="accordion-toggle-sub"

Packages

Create packages for:

• Homebrew
• Nixpkgs

Body example

Is there an example of what should be placed in the body file, what format should the file be in?

Feature:Randomize source addresses

Could you please add a feature to allow a CIDR to be specified as the source network to generate traffic. The IP addresses that are part of the CIDR provided would then be added to the interface using the ifconfig command.

An alternate method would to allow the user to specify the list of IP addresses in a file and then have the user also configure their interface with this list of IP addresses (or have the program generate traffic based on all of the IP's configured on a given interface).

I have a use-case where the security devices I test use a src/dst hashing algorithm and will load share the traffic based on these flows. I could do this based on destination address however this would not simulate a real life scenario such as hundreds of users accessing a single web server.

Support Websockets

Drones with embedded web servers typically use websockets. Shouldn't the prince of the Saiyans be able to attack enemies in the air?

Incremental reporters

With the previous work on making most of the data pipeline stream results as they come, it would be great to have a some new reporters which incrementally reflect new incoming data.

Feature: Different POST payloads for different endpoints

Vegeta does not seem to support different POST request bodies per test. Correct me, if I'm wrong. I have to test multiple endpoints that do no share the same request payload. What if vegeta would support a target file structure like this:

POST http://example.com/create/1 {"test":1}
POST http://example.com/create/2 {"test":2}

and only use the same body if body option is provided?

Perpetual attack

It may be useful to have an attack run for an indefinite duration. CTRL+C must be caught and initiate graceful attack termination. This feature would go well along with the proposed #53.

Dump full requests and reponses

Hi Tomás,

For debugging purposes is it possible to dump full requests and responses?

Many Thanks

Aidy

Use `go generate` for packaging dygraphs

Currently the dygraphs plotting library is being packaged in the binaries with https://github.com/jteeuwen/go-bindata.

With the advent of the go generate in Go 1.4, this process can likely be improved with automation.

ssl client certs

I wondered if you'd ever thought of supporting ssl client certs?

Linux builds can't be opened

I ran:

wget https://dl.dropboxusercontent.com/u/83217940/vegeta-linux-amd64.tar.gz
tar xvzf vegeta-linux-amd64.tar.gz

And my terminal said:

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

I also tried with the 32-bit version, and it didn't work either.

Can't set Host header

golang's net/http library seems to ignore a Host header and only uses the value set in request struct or extracts it from url.Host. In order to be able to set a flag like -header="Host: foo.bar" it's necessary to treat that that header different than others.

This works for me:

diff --git a/lib/targets.go b/lib/targets.go
index 9b9739a..b3d70b2 100644
--- a/lib/targets.go
+++ b/lib/targets.go
@@ -63,8 +63,12 @@ func (t Targets) SetHeader(header http.Header) {
        for _, target := range t {
                target.Header = make(http.Header, len(header))
                for k, vs := range header {
-                       target.Header[k] = make([]string, len(vs))
-                       copy(target.Header[k], vs)
+                       if k == "Host" {
+                               target.Host = vs[0]
+                       } else {
+                               target.Header[k] = make([]string, len(vs))
+                               copy(target.Header[k], vs)
+                       }
                }
        }
 }

No idea why the copy of the headers is necessary, it might be necessary for the Host field as well then.

Better usage output

The output of vegeta -h can be largely improved.

Experiment various scenarios with the Go execution tracer

https://docs.google.com/document/d/1FP5apqzBgr7ahCCgFO-yoVhk4YZrNIDNf9RybngBc14/pub

Error in go 1.3.1

Hi I have following error in golang 1.3.1

../go/src/github.com/tsenart/vegeta/lib/metrics.go:7: import /home/jacekwysocki/go/pkg/linux_amd64/github.com/bmizerany/perks/quantile.a: object is [linux amd64 go1.2.2 X:none] expected [linux amd64 go1.3.1 X:precisestack]

is there any workaround for this ?

out of memory with big big input file

fatal error: runtime: out of memory

goroutine 16 [running]:
runtime.throw(0x890077)
/usr/src/go/src/pkg/runtime/panic.c:520 +0x69 fp=0x7f1595b636d8 sp=0x7f1595b636c0
runtime.SysMap(0xc2dee20000, 0x1c040000, 0x574f00, 0x89b998)
/usr/src/go/src/pkg/runtime/mem_linux.c:147 +0x93 fp=0x7f1595b63708 sp=0x7f1595b636d8
runtime.MHeap_SysAlloc(0x8a7980, 0x1c040000)
/usr/src/go/src/pkg/runtime/malloc.goc:616 +0x15b fp=0x7f1595b63760 sp=0x7f1595b63708
MHeap_Grow(0x8a7980, 0xe020)
/usr/src/go/src/pkg/runtime/mheap.c:319 +0x5d fp=0x7f1595b637a0 sp=0x7f1595b63760
MHeap_AllocLocked(0x8a7980, 0xe01e, 0x0)
/usr/src/go/src/pkg/runtime/mheap.c:222 +0x379 fp=0x7f1595b637e0 sp=0x7f1595b637a0
runtime.MHeap_Alloc(0x8a7980, 0xe01e, 0x10100000000)
/usr/src/go/src/pkg/runtime/mheap.c:178 +0x7b fp=0x7f1595b63808 sp=0x7f1595b637e0
largealloc(0x0, 0x7f1595b638b8)
/usr/src/go/src/pkg/runtime/malloc.goc:224 +0xa2 fp=0x7f1595b63850 sp=0x7f1595b63808
runtime.mallocgc(0x1c03c000, 0x6bcfe1, 0x0)
/usr/src/go/src/pkg/runtime/malloc.goc:169 +0xb6 fp=0x7f1595b638b8 sp=0x7f1595b63850
growslice1(0x692900, 0xc2c8780000, 0x59a580, 0x59a580, 0x59a581, 0x7f1595b63978)
/usr/src/go/src/pkg/runtime/slice.goc:125 +0x107 fp=0x7f1595b63908 sp=0x7f1595b638b8
runtime.growslice(0x692900, 0xc2c8780000, 0x59a580, 0x59a580, 0x1, 0x0, 0x0, 0x0)
/usr/src/go/src/pkg/runtime/slice.goc:76 +0xbb fp=0x7f1595b63950 sp=0x7f1595b63908
github.com/tsenart/vegeta/lib.NewTargets(0xc248086000, 0x31a43867, 0x3ffffe00, 0x896080, 0x0, 0x0, 0xc20800f080, 0x0, 0x0, 0x0, ...)
/go/src/github.com/tsenart/vegeta/lib/targets.go:62 +0x5e9 fp=0x7f1595b63b40 sp=0x7f1595b63950
main.attack(0xc208052280, 0x0, 0x0)
/go/src/github.com/tsenart/vegeta/attack.go:101 +0x3c0 fp=0x7f1595b63dd8 sp=0x7f1595b63b40
main.func·001(0xc20800e020, 0x1, 0x1, 0x0, 0x0)
/go/src/github.com/tsenart/vegeta/attack.go:40 +0x73 fp=0x7f1595b63e18 sp=0x7f1595b63dd8
main.main()
/go/src/github.com/tsenart/vegeta/main.go:37 +0x409 fp=0x7f1595b63f50 sp=0x7f1595b63e18
runtime.main()
/usr/src/go/src/pkg/runtime/proc.c:247 +0x11a fp=0x7f1595b63fa8 sp=0x7f1595b63f50
runtime.goexit()
/usr/src/go/src/pkg/runtime/proc.c:1445 fp=0x7f1595b63fb0 sp=0x7f1595b63fa8
created by _rt0_go
/usr/src/go/src/pkg/runtime/asm_amd64.s:97 +0x120

goroutine 19 [finalizer wait]:
runtime.park(0x417d30, 0x8957b8, 0x893a09)
/usr/src/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x8957b8, 0x893a09)
/usr/src/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
/usr/src/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
/usr/src/go/src/pkg/runtime/proc.c:1445

Add disable-keepalive flag

The service I'm testing almost exclusively deals with short-lived, single request connections. With no option to disable keepalive I cannot run tests which properly simulate my production environment.

Would you accept a PR with a -disable-keepalive flag?