Load testing at cloud-scale. Serverless & distributed out-of-the-box. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module. Never fail to scale with Artillery!
Home Page: https://www.artillery.io
License: Mozilla Public License 2.0
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
I realise there is a test for this that seems to pass, but given my CSV:
abcde,Mita,20554,Novick,Septfontaines,1111,[email protected],Tinactin,Andrea Snyder,[email protected]
1243fsdfs,Browsezoom,385,Logan,Malkā,4444,[email protected],SORE MUSCLE AND JOINT PAIN RELIEF,Lawrence Evans,[email protected]
gdfgdfg,Flashset,9451,Sage,Amarillo,79165,[email protected],Morphine Sulfate,Eugene Bell,[email protected]
and my config block:
"payload": {
"path": "./data2.csv",
"fields": ["registration","name","building","street","city","postcode","email","branch","fullName","fullEmail"]
}
My request is:
{
"loop": [
{
"post": {
"url": "/v1/xxyyzz/abcde",
"json": {
"registration": "{{ registration }}",
"name": "{{ name }}",
"contact": {
"address": {
"building": "{{ building }}",
"street": "{{ street }}",
"line1": "1th 4th",
"city": "{{ city }}",
"postCode": "1234",
"country": "DK"
},
"phoneNumber": "12345678",
"email": "{{ email }}"
}
},
"capture": {
"json": "$._id",
"as": "someId"
}
}
}
],
"count": 1
}
I'm having no luck at all - unfortunately I can't see a way to get good debug information about the requests i'm sending (http:request would be nice) but my API is telling me that 'registration' is blank each time, and if I hardcode it, the next entry is blank, and so on.
As discussed on gitter, environment specific target urls are not picked up.
For example:
"config": {
"target": "xyz"
"environments": {
"perf": {
"target": "http://my-api.example.net"
}
}
}
Then running with '-e perf' results in all requests actually being prefixed with: xyz rather than the environment's URL.
Leaving the config.target element out entirely results in the url being prefixed with undefined
Expected behaviour is that the URL /v1/user would become http://my-api.example.net/v1/user rather than xyz/v1/user.
$ minigun quick -d 600 -r 200 -o out-`date +"%s"` http://127.0.0.1:8124
Log file: out-1450455607.json
Phase undefined started - duration: 600s
$ minigun -V
1.2.14
-o option can be passed in in quick mode, but is never referenced (and therefore never used).
e.g. ~/.minigunconfig
This will allow users to enable certain plugins by default, e.g.: minigun-plugin-blessed
Hello, awesome Artillery contributors! (tagged in this Issue)
I have a favor to ask. To help grow the community around this project and to help ensure its long-term success, I'd like to release all future versions of Artillery under the Mozilla Public License v2 (MPLv2).
Why MPLv2?
In a nutshell, MPLv2 sits between strong copyleft licenses like GPL and permissive licenses like ISC/MIT and provides the best of both worlds:
artillery-core directly, and does not apply to the CLI) or mix it with code under most other open-source licensesMPLv2 is business-friendly, and helps protect the project and its contributors against hostile forks, being re-released as a closed-source product, or patent-related lawsuits.
Some further reading:
If you're okay with your contribution to be distributed under MPLv2 going forward, please reply to this thread with a "yes". 🎈
Any questions, please ask away.
Cheers
Is there support (or planned) for distributed load testing?
We currently use Locust which handles this quite nicely using master / slave mode:
locust -f my_locustfile.py --master
locust -f my_locustfile.py --slave --master-host=192.168.0.14
http://docs.locust.io/en/latest/running-locust-distributed.html
I've edited the call stack a bit to remove personal info
artillery quick --duration 10 --rate 20 https://example.com
fs.js:549
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: ENOENT: no such file or directory, open 'C:\tmp\artillery_quick_script_20160302_223210.json'
at Error (native)
at Object.fs.openSync (fs.js:549:18)
at Object.fs.writeFileSync (fs.js:1155:15)
at Command.quick (<user_home>\AppData\Roaming\npm\node_modules\artillery\lib\commands\quick.js:79:6)
at Command.listener (<user_home>\AppData\Roaming\npm\node_modules\artillery\node_modules\commander\index.js:301:8)
at emitTwo (events.js:87:13)
at Command.emit (events.js:172:7)
at Command.parseArgs (<user_home>\AppData\Roaming\npm\node_modules\artillery\node_modules\commander\index.js:615:12)
at Command.parse (<user_home>\AppData\Roaming\npm\node_modules\artillery\node_modules\commander\index.js:458:21)
at Object.<anonymous> (<user_home>\AppData\Roaming\npm\node_modules\artillery\bin\artillery:58:9)
Run a test with DEBUG=http,http:response together.
Results: We print a list of requests and then a list of responses.
Expected Behavior: We should print the request and subsequent response together so it's easier to match up our response to our request .
A few rather embarrassing bugs have been shipped in the last couple of months which could have been avoided with more rigorous testing. Test coverage used to be at 90%+ but it isn't anymore.
To help prevent shipping more bugs in the future we should:
At the moment it only works for the payload, and should be helpful if working with variables as well.
Thank you
Hi !
I've been using Minigun to send requests with base64 img in request params. But i've got the following error when I'm launching minigun:
[minigum_process_out_of_memory.zip](https://github.com/shoreditch-ops/minigun/files/66994/minigum_process_out_of_memory.zip)
Phase undefined (Test POST Requests) started - duration: 2000s
<--- Last few GCs --->
8991 ms: Scavenge 1399.0 (1454.4) -> 1399.0 (1454.4) MB, 1.8 / 0 ms [allocation failure].
8993 ms: Scavenge 1399.0 (1454.4) -> 1399.0 (1454.4) MB, 1.8 / 0 ms [allocation failure].
8995 ms: Scavenge 1399.0 (1454.4) -> 1399.0 (1454.4) MB, 1.8 / 0 ms [allocation failure].
9958 ms: Mark-sweep 1399.0 (1454.4) -> 1399.0 (1454.4) MB, 963.7 / 0 ms [last resort gc].
10906 ms: Mark-sweep 1399.0 (1454.4) -> 1399.0 (1454.4) MB, 947.5 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x1ba9c0137339 <JS Object>
2: squashTokens [/home/arthur/.nvm/versions/node/v4.1.1/lib/node_modules/minigun/node_modules/minigun-core/node_modules/mustache/mustache.js:234] [pc=0x36eeedddf82] (this=0x1ba9c01b63c1 <JS Global Object>,tokens=0x23bc9027b409 <JS Array[15085174]>)
3: parseTemplate(aka parseTemplate) [/home/arthur/.nvm/versions/node/v4.1.1/lib/node_modules/minigun/node_modules/minigun-core/node_modules/...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abandon (core dumped)
I've joined a zip file with the minigun params i use.
We should add an AMQP engine to make Artillery even more useful for teams working on microservice-oriented architectures where RabbitMQ is often employed as microservice glue.
The existing WebSocket engine (engine_ws.js) would be a good starting point.
In our unit and integration tests, we use http://chancejs.com a lot, for reasons of variation testing, data realism, and general amusement.
I'd like to see the same sort of support in Artillery so that we can do something like
"scenarios": [
{
"flow": [
{
"loop": [
{"post": {
"url": "/create-user",
"json": {
"email": "{{ chance.email() }}",
"password": "{{ chance.string(length: 10) }}"
}
}}
],
"count": 100
}
]
}
]
The format of the chance inserts is undetermined, can't immediately think of a nice way to do it.
Right now I'm using mockaroo (some website I found which generates CSVs with chance-like data) to generate CSVs, but it's a pay-for product and it's still very static.
Thoughts?
Hi, I think it will be great if minigun run command produces cpu & memory usage report.
I am testing this by incorporating usage module into minigun-core module and supplying pid in the JSON file that 'minigun run' command will read. If pid is not given, run command should work without collecting cpu & memory data. Would appreciate your opinion on this!
This would be nice as it would be possible to implement annotations on graphs, to show when the tests ramp up / down.
The phases in this graph are:
"phases": [
{ "duration": 120, "arrivalRate": 1, "name": "Search ramp-up" },
{ "duration": 60, "arrivalRate": 5, "name": "Search ramp-up" }
],This seems to cause the following:
-/home/me/git/artillery-test/node_modules/artillery/node_modules/artillery-core/lib/engine_util.js:69
let match = o.match(funcCallRegex);
^
TypeError: Cannot read property 'match' of undefined
at template (/home/me/git/artillery-test/node_modules/artillery/node_modules/artillery-core/lib/engine_util.js:69:18)
at f (/home/me/git/artillery-test/node_modules/artillery/node_modules/artillery-core/lib/engine_http.js:37:32)
at fn (/home/me/git/artillery-test/node_modules/async/lib/async.js:746:34)
at /home/me/git/artillery-test/node_modules/async/lib/async.js:1213:16
at /home/me/git/artillery-test/node_modules/async/lib/async.js:166:37
at /home/me/git/artillery-test/node_modules/async/lib/async.js:706:43
at /home/me/git/artillery-test/node_modules/async/lib/async.js:167:37
at /home/me/git/artillery-test/node_modules/async/lib/async.js:1209:30
at Request.requestCallback [as _callback] (/home/me/git/artillery-test/node_modules/artillery/node_modules/artillery-core/lib/engine_http.js:136:14)
at Request.self.callback (/home/me/git/artillery-test/node_modules/artillery/node_modules/artillery-core/node_modules/request/request.js:197:22)It's great library but i need some examples with Socket.io. Can you help me with this?
Something that I want to do is just make the html file available for downloading from my CI environment, and not try to open it in a browser.
Opening the browser means my build never ends, and fails.
I propose passing -o/--output to the report command, as with the run command, and if specified, output to a file and don't try to open the report.
I am looking at creating a RabbitMQ engine for Minigun. At the moment I am finding myself having to fork minigun-core to add what I need, do you plan to implement support for creating engines outside of minigun-core?
Minigun should provide the ability to generate graphical reports from the JSON reports created by minigun run.
Requirements:
To generate a report:
minigun report minigun_report_20150806_115304.json
This will create the HTML file in current working directory and open it in the default browser.
I am trying to get a loop step to work but am getting the following exception:
/node_modules/artillery/node_modules/artillery-core/lib/engine_util.js:44
let match = o.match(funcCallRegex);
^
TypeError: Cannot read property 'match' of undefined
at template (/node_modules/artillery/node_modules/artillery-core/lib/engine_util.js:44:18)
at f (/node_modules/artillery/node_modules/artillery-core/lib/engine_http.js:26:32)
at fn (/node_modules/artillery/node_modules/async/lib/async.js:746:34)
at /node_modules/artillery/node_modules/async/lib/async.js:1213:16
at /node_modules/artillery/node_modules/async/lib/async.js:166:37
at /node_modules/artillery/node_modules/async/lib/async.js:706:43
at /node_modules/artillery/node_modules/async/lib/async.js:167:37
at Immediate._onImmediate (/node_modules/artillery/node_modules/async/lib/async.js:1206:34)
at processImmediate [as _immediateCallback] (timers.js:368:17)
I have literally tried the example file from your unit tests folder to make sure there's no syntax error: https://github.com/shoreditch-ops/artillery-core/blob/master/test/scripts/loop.json
Findings:
The step function (in engine_http.js) tries to pass params.url to the template function (in engine_util.js). However params.url is undefined in the case of loops as they are arrays, not objects: [ { get: { url: '/' } } ]
Am I doing something wrong? Has loop been fully implemented yet or should loop be an object with a single step rather than an array with a list of steps?
This is conflicting with the --help information:
dino [options] Show dinosaur of the day
$ minigun dino
__________
< minigun! >
----------
\
\
__
/ _)
.-^^^-/ /
__/ /
<__.|_|-|_|
$ minigun dino
__________
< minigun! >
----------
\
\
__
/ _)
_/\/\/\_/ /
_| /
_| ( | ( |
/__.-'|_|--|_|
Link to https://github.com/shoreditch-ops/artillery-plugin-statsd is placed in /docs/plugins.md but artillery-plugin-statsd does not exist. 'minigun-plugin-statsd' project name must be replaced by 'artillery-plugin-statsd', along with the other plugins.
Is there any possibility to specify a longer timeout than 10s, which seems to be the current limit? Our server (and third-party services behind it) does some heavy lifting with payments etc and 10s doesn't quite cover it for us.
I might be configuring this wrong, but if I configure a loop and omit the 'count' variable to have the loop repeat indefinitely, requests gets reported as 0 and no stats are recorded for request latency. If I add the count variable then things seem to work as expected
Any guidelines for a PR on this?
Hello:
I am trying to user Artillery to load test my node application. I am running Artillery on my development machine hence can see the logs which are generated when even any api call is made from other application.
This is output which I get from after running artillery on my app
Scenarios launched: 26070
Scenarios completed: 78
Number of requests made: 78
RPS: 82.18
Request latency:
min: 414.9
max: 9809.6
median: 3550.8
p95: 9245.4
p99: 9755.3
Scenario duration:
min: 442.3
max: 9812
median: 3554.5
p95: 9249.3
p99: 9757.6
Codes:
200: 78
Errors:
ETIMEDOUT: 9412
ESOCKETTIMEDOUT: 30
ECONNREFUSED: 16550
Clearly there are Huge number of ETIMEDOUT and ECONNREFUESED count. While in the log I do not see any errors.
If It helps this is my json file
{
"config": {
"target": "http://localhost:8090/api/v1",
"phases": [{
"duration": 60,
"arrivalRate": 5,
"rampTo": 10
}, {
"duration": 120,
"arrivalRate": 10,
"rampTo": 50
}, {
"duration": 120,
"arrivalRate": 5,
"rampTo": 500
}]
},
"scenarios": [{
"name": "Test",
"flow": [{
"post": {
"url": "/users/",
"json": {
"name": "test"
},
"capture": {
"json": "$.data.id",
"as": "id"
}
}
}]
}]
}
I am not sure what I am doing wrong. Any idea what can be wrong?
It would be nice to be able to assemble URLs from variables in scenarios, therefore would be nice to have target available as a variable by default.
With artillery, this is what I get from a simple GET scenario to my API:
all scenarios completed
Complete report @ 2016-04-13T16:06:16.659Z
Scenarios launched: 4719
Scenarios completed: 3551
Number of requests made: 3551
RPS: 29.58
Request latency:
min: 2
max: 9999.3
median: 3743.1
p95: 6865.1
p99: 9343
Scenario duration:
min: 2.5
max: 9999.7
median: 3743.6
p95: 6865.7
p99: 9343.7
Codes:
200: 3551
Errors:
ESOCKETTIMEDOUT: 29
ETIMEDOUT: 1139
Using loadtest, this is what I get for the same exact API (and from the same dev box)
loadtest -n 3551 -c 50 --rps 50 http://mydomain.com/myApi/hi
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Target URL: http://mydomain.com/myApi/hi
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Max requests: 3551
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Concurrency level: 50
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Agent: none
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Requests per second: 50
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Completed requests: 3551
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Total errors: 0
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Total time: 71.009830928 s
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Requests per second: 50
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Total time: 71.009830928 s
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO Percentage of the requests served within a certain time
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO 50% 2 ms
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO 90% 3 ms
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO 95% 3 ms
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO 99% 3 ms
[Wed Apr 13 2016 12:10:06 GMT-0400 (EDT)] INFO 100% 18 ms (longest request)
Not only is my RPS higher, my latency/response time is MUCH MUCH lower. What's going on?
Especially considering this applications goal is to load test something I would recommend against using Request module, its bloated and less performant than something like Wreck
tomg@ttlpenguin ~/ (minigun●●)(4.2.4)$ DEBUG=rascal:Publication minigun run -e local test/load/searches.json | grep 'Publishing 215' | uniq -c
/usr/local/lib/node_modules/minigun/node_modules/cli/cli.js:1147
stream.clearLine();
^
TypeError: stream.clearLine is not a function
at null._repeat (/usr/local/lib/node_modules/minigun/node_modules/cli/cli.js:1147:16)
at wrapper [as _onTimeout] (timers.js:275:19)
at Timer.listOnTimeout (timers.js:92:15)
Hello. Loving this tool.
How can I configure it to not validate SSL certs. I am using self-signed certs in our staging environment.
All requests are errors - DEPTH_ZERO_SELF_SIGNED_CERT.
Thank you!
I am trying to install minigun on my Mac with OSX 10.10.5 but it failed. There are several error message according to bufferutil:
> [email protected] install /Users/xxx/.nvm/versions/node/v4.1.0/lib/node_modules/minigun/node_modules/bufferutil
> node-gyp rebuild
build:minigun-core ▐ ╢███████████████████████████████████ CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
In file included from ../src/bufferutil.cc:16:
../../nan/nan.h:261:25: error: redefinition of '_NanEnsureLocal'
NAN_INLINE v8::Local<T> _NanEnsureLocal(v8::Local<T> val) {
^
../../nan/nan.h:256:25: note: previous definition is here
NAN_INLINE v8::Local<T> _NanEnsureLocal(v8::Handle<T> val) {
^
../../nan/nan.h:661:13: error: no member named 'smalloc' in namespace 'node'
, node::smalloc::FreeCallback callback
~~~~~~^
../../nan/nan.h:672:12: error: no matching function for call to 'New'
return node::Buffer::New(v8::Isolate::GetCurrent(), data, size);
^~~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/node_buffer.h:31:40: note: candidate function not
viable: no known conversion from 'uint32_t' (aka 'unsigned int') to 'enum encoding' for 3rd
argument
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
^
/Users/xxx/.node-gyp/4.1.0/include/node/node_buffer.h:43:40: note: candidate function not
viable: 2nd argument ('const char *') would lose const qualifier
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
^
/Users/xxx/.node-gyp/4.1.0/include/node/node_buffer.h:28:40: note: candidate function not
viable: requires 2 arguments, but 3 were provided
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate, size_t length);
^
/Users/xxx/.node-gyp/4.1.0/include/node/node_buffer.h:36:40: note: candidate function not
viable: requires 5 arguments, but 3 were provided
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
^
In file included from ../src/bufferutil.cc:16:
../../nan/nan.h:676:12: error: no viable conversion from 'v8::MaybeLocal<v8::Object>' to
'v8::Local<v8::Object>'
return node::Buffer::New(v8::Isolate::GetCurrent(), size);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:210:7: note: candidate constructor
(the implicit copy constructor) not viable: no known conversion from
'v8::MaybeLocal<v8::Object>' to 'const v8::Local<v8::Object> &' for 1st argument
class Local {
^
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:210:7: note: candidate constructor
(the implicit move constructor) not viable: no known conversion from
'v8::MaybeLocal<v8::Object>' to 'v8::Local<v8::Object> &&' for 1st argument
class Local {
^
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:214:13: note: candidate template ignored: could
not match 'Local' against 'MaybeLocal'
V8_INLINE Local(Local<S> that)
^
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:326:13: note: candidate template ignored: could
not match 'S *' against 'v8::MaybeLocal<v8::Object>'
V8_INLINE Local(S* that)
^
In file included from ../src/bufferutil.cc:16:
../../nan/nan.h:683:26: error: no member named 'Use' in namespace 'node::Buffer'
return node::Buffer::Use(v8::Isolate::GetCurrent(), data, size);
~~~~~~~~~~~~~~^
In file included from ../src/bufferutil.cc:7:
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:221:5: error: assigning to
'v8::Primitive *volatile' from incompatible type 'v8::Value *'
TYPE_CHECK(T, S);
^~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:180:37: note: expanded from macro 'TYPE_CHECK'
*(static_cast<T* volatile*>(0)) = static_cast<S*>(0); \
^ ~~~~~~~~~~~~~~~~~~
../../nan/nan.h:414:12: note: in instantiation of function template specialization
'v8::Local<v8::Primitive>::Local<v8::Value>' requested here
return NanEscapeScope(NanNew(v8::Undefined(v8::Isolate::GetCurrent())));
^
../../nan/nan.h:398:30: note: expanded from macro 'NanEscapeScope'
# define NanEscapeScope(val) scope.Escape(_NanEnsureLocal(val))
^
In file included from ../src/bufferutil.cc:7:
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:221:5: error: assigning to
'v8::Boolean *volatile' from incompatible type 'v8::Value *'
TYPE_CHECK(T, S);
^~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:180:37: note: expanded from macro 'TYPE_CHECK'
*(static_cast<T* volatile*>(0)) = static_cast<S*>(0); \
^ ~~~~~~~~~~~~~~~~~~
../../nan/nan.h:424:12: note: in instantiation of function template specialization
'v8::Local<v8::Boolean>::Local<v8::Value>' requested here
return NanEscapeScope(NanNew(v8::True(v8::Isolate::GetCurrent())));
^
../../nan/nan.h:398:30: note: expanded from macro 'NanEscapeScope'
# define NanEscapeScope(val) scope.Escape(_NanEnsureLocal(val))
^
In file included from ../src/bufferutil.cc:7:
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:221:5: error: assigning to
'v8::Function *volatile' from incompatible type 'v8::Value *'
TYPE_CHECK(T, S);
^~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:180:37: note: expanded from macro 'TYPE_CHECK'
*(static_cast<T* volatile*>(0)) = static_cast<S*>(0); \
^ ~~~~~~~~~~~~~~~~~~
../../nan/nan.h:1514:12: note: in instantiation of function template specialization
'v8::Local<v8::Function>::Local<v8::Value>' requested here
return NanEscapeScope(NanNew(handle)->Get(kCallbackIndex)
^
../../nan/nan.h:398:30: note: expanded from macro 'NanEscapeScope'
# define NanEscapeScope(val) scope.Escape(_NanEnsureLocal(val))
^
In file included from ../src/bufferutil.cc:7:
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:221:5: error: assigning to
'v8::Object *volatile' from incompatible type 'v8::Value *'
TYPE_CHECK(T, S);
^~~~~~~~~~~~~~~~
/Users/xxx/.node-gyp/4.1.0/include/node/v8.h:180:37: note: expanded from macro 'TYPE_CHECK'
*(static_cast<T* volatile*>(0)) = static_cast<S*>(0); \
^ ~~~~~~~~~~~~~~~~~~
../../nan/nan.h:1632:12: note: in instantiation of function template specialization
'v8::Local<v8::Object>::Local<v8::Value>' requested here
return NanEscapeScope(handle->Get(NanNew(key)).As<v8::Object>());
^
../../nan/nan.h:398:30: note: expanded from macro 'NanEscapeScope'
# define NanEscapeScope(val) scope.Escape(_NanEnsureLocal(val))
^
9 errors generated.
make: *** [Release/obj.target/bufferutil/src/bufferutil.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/Users/xxx/.nvm/versions/node/v4.1.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:270:23)
gyp ERR! stack at emitTwo (events.js:87:13)
gyp ERR! stack at ChildProcess.emit (events.js:172:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Darwin 14.5.0
gyp ERR! command "/Users/xxx/.nvm/versions/node/v4.1.0/bin/node" "/Users/xxx/.nvm/versions/node/v4.1.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/xxx/.nvm/versions/node/v4.1.0/lib/node_modules/minigun/node_modules/bufferutil
gyp ERR! node -v v4.1.0
gyp ERR! node-gyp -v v3.0.1
gyp ERR! not ok
I have checked bufferutil's project page here and seems that they have a workaround by adding a npm-shrinkwrap.json in the project root.
Artillery does not always recognize valid TLS certificates. For example:
$ artillery quick -d 10 -r 5 https://artillery.io
Scenarios launched: 49
Scenarios completed: 0
Number of requests made: 0
RPS: 4.73
Request latency:
min: Infinity
max: -Infinity
median: NaN
p95: NaN
p99: NaN
Scenario duration:
min: Infinity
max: -Infinity
median: NaN
p95: NaN
p99: NaN
Errors:
UNABLE_TO_VERIFY_LEAF_SIGNATURE: 49
The issue is that Node.js uses its own certificate bundle, which does not include many CAs. To fix, a custom bundle needs to be used by the HTTP engine.
It would be useful to have some support for using regular expressions instead of JSONPath or XPath to parsing and capturing responses, for situations where the response body is neither JSON nor XML.
Minigun needs to provide better visibility into what's happening under the hood while a test is running.
This is useful for developers that want to contribute new features or fix bugs, and for users while writing and debugging scenarios.
I want to be able to run the same scenario across multiple environments. I can easily copy the json file per environment or use the json file as template for all scenarios I want to run.
e.g. the target might be
Current output of artillery report is quite useful, but there are many ways in which the reports can be improved further to help analyse performance of the systems under test.
Specifically, with this change in core it's now possible to show fine-grained request latencies: artilleryio/artillery-core@fe9e0b6
There are other enhancements that can be made for v1.42.0 - this is the thread to collect them all.
https://artillery.io/minigun/ contains 4 404s
Each of these links generate a 404
Specify multiple load-phases
Set custom headers
Use payload files for POST data
Set expectations on response latency
I didn't see the source for https://artillery.io/minigun/ on github so I can't fix the problem :(
For maximum performance Artillery should take advantage of every CPU available. 🔥 🔥
Hi @hassy the 1.3.1 version you pushed earlier has a rather nasty breaking bug in it.
➜ repro minigun run repro.json
/usr/local/lib/node_modules/minigun/node_modules/minigun-core/lib/runner.js:67
fields: script.config.payload.fields,
^
TypeError: Cannot read property 'fields' of undefined
at runner (/usr/local/lib/node_modules/minigun/node_modules/minigun-core/lib/runner.js:67:40)
at /usr/local/lib/node_modules/minigun/lib/commands/run.js:149:16
at /usr/local/lib/node_modules/minigun/node_modules/async/lib/async.js:52:16
at Immediate._onImmediate (/usr/local/lib/node_modules/minigun/node_modules/async/lib/async.js:1206:34)
at processImmediate [as _immediateCallback] (timers.js:383:17)
Here is a simple reproduction config:
{
"config": {
"target": "https://google.com",
"phases": [
{ "duration": 120, "arrivalRate": 10 }
]
},
"scenarios": [
{ "get": { "url": "/ping" } }
]
}
The bug appears to be because of this line: https://github.com/shoreditch-ops/minigun/blob/46ddfd1c9f3efa57a171287719c44216020ca3be/lib/commands/run.js#L124 causing this falsity check: https://github.com/shoreditch-ops/minigun-core/blob/380f5c76f86ea6ca1d14a861bf819e3651373401/lib/runner.js#L63 to incorrectly enter the block. Then on L67 referencing payload.fields fails because payload does not actually exist.
Each scenario in my test needs to have a unique orderNumber variable. My orderNumbers are read in from CSV file, but because these are picked at random the same orderNumber can (and does) get re-used. Is there any way that variables from CSV can be looped through in sequence rather than picked at random?
Right now, artillery quick is just a thin layer on top of the run command. Several ways in which it can be improved:
wrk / vegeta (set number of requests to send, set concurrency, set target RPS).quick as opposed to run, e.g. displaying scenario stats does not make sense in the context (the user didn't specify any scenarios, rather the command generated one behind the scenes)I am using a header to pass on JWT token which is used for authenticating/authorizing the application user. When user authenticates first providing his credentials, he will get a JWT token created and sent using a header to client. This header has to be captured and stored as a variable.
This token has to be sent to server on subsequent requests. As per my understanding looking at the code, this is not possible as of now!. Can somebody provide me insights into feasibility of this problem? Or something to work as a work around?
Note: I see that there is a afterResponse method callback which I can register to get the response in callback. I can use it, but I don't know how to store a variable from third party callback into artillery context.
It would be nice to have support for Postman collections in Artillery.
Minigun should have support for XMPP/Jabber.
Trying figuring out how I can provide a filename in .json config files for the output file generated in reports
I've found in src files this:
lib/commands/run.js
const defaultOptions = require('rc')('artillery');
...
if (defaultOptions.output) {
logfile = moment().format(defaultOptions.output);
} else {
logfile = moment().format('[artillery_report_]YMMDD_HHmmSS[.json]');
}
How can i set this?
Is this usable from config file?
Originally prompted by: #1
It would be nice to have first-class Socket.io support in Artillery.
(Closes #9)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.