Giter Site home page Giter Site logo

nodeload's Introduction

CURRENT STATE

In case it isn't obvious, this project is no longer being actively developed or maintained.

Feel free to fork or take the idea and run with it.

INSTALLING

Make sure node.js is installed. Then install nodeload:

  1. Using npm:

     curl http://npmjs.org/install.sh | sh       # install npm if not already installed
     npm install nodeload
    
  2. From source:

     git clone git://github.com/benschmaus/nodeload.git
     cd nodeload
     npm link    # optional. enables require('nodeload/<module>') instead of require('./lib/<module>').
    
  3. Or as a single file (this does not install the nl.js tool):

     wget https://github.com/benschmaus/nodeload/raw/master/nodeload.js
    

NODELOAD

nodeload is a collection of independent node.js modules for load testing HTTP services.

As a developer, you should be able to write load tests and get informative reports without having to learn another framework. You should be able to build by example and selectively use the parts of a tool that fit your task. Being a library means that you can use as much or as little of nodeload as makes sense, and you can create load tests with the power of a full programming language. For example, if you need to execute some function at a given rate, just use the 'nodeload/loop' module, and write the rest yourself.

In addition, nodeload is built for operability. It can always be deployed by simply copying the single file, nodeload.js.

Look for examples in the examples/ directory and in test cases prefixed with "example" in test/. Here are simple examples of each module:

nl

nl is an Apache Bench (ab) like command line tool for running tests quickly. See the nl documentation for details.

$ examples/test-server.js &    # starts a simple server on port 9000 to load test
$ ./nl.js -c 10 -n 10000 -i 2 localhost:9000

will send 10,000 queries to http://localhost:9000 using 10 connections. Statistics are printed to the console and graphs can be seen at http://localhost:8000/.

nodeload

The nodeload module is the primary interface for creating load tests. It includes all the other modules described below, so if you require('nodeload'), you don't need to require() any of the other ones. Look at the examples in examples/loadtesting.ex.js and examples/riaktest.ex.js or read the nodeload module documentation.

var nl = require('nodeload');
var loadtest = nl.run({
    host: 'localhost',
    port: 9000,
    timeLimit: 60,
    targetRps: 500,
    requestGenerator: function(client) {
        var request = client.request('GET', "/" + Math.floor(Math.random()*10000));
        request.end();
        return request;
    }
});
loadtest.on('end', function() { console.log('Load test done.'); });

remote

The remote module provides a mechanism for running a distributed load test. See examples/remotetesting.ex.js and examples/remote.ex.js for examples or read the remote module documentation.

Start slave instances:

$ HTTP_PORT=10001 ./nodeload.js  # start a local slave instance on :10001
$ HTTP_PORT=10002 ./nodeload.js  # start a 2nd slave instance on :10002

Create the distributed load test:

var nl = require('nodeload/remote');
var cluster = new nl.LoadTestCluster('localhost:8000', ['localhost:8002', 'localhost:8001']);
cluster.run({
    host: 'localhost',
    port: 9000,
    timeLimit: 60,
    targetRps: 500,
    requestGenerator: function(client) {
        var request = client.request('GET', "/" + Math.floor(Math.random()*10000));
        request.end();
        return request;
    }
});
cluster.on('end', function() { console.log('Load test done.'); });

stats

The stats module provides implementations of various statistics objects, like Histograms and Accumulators, and functions, like randomString(), and nextGaussian(). See the stats module documentation.

var stats = require('nodeload/stats');
var histogram = new stats.Histogram();
for (var i = 0; i < 1000; i++) 
    histogram.put(Math.abs(Math.floor(stats.nextGaussian())));
console.log('Mean: ' + histogram.mean() + ', 99%: ' + histogram.percentile(0.99));

will output "Mean: 0.852, 99%: 3".

monitoring

The monitoring module provides a way to track runtime statistics for code that is run concurrently. See test/monitoring.test.js for examples or read the monitoring module documentation.

var monitoring = require('nodeload/monitoring');
var monitor = new monitoring.Monitor('runtime');
function asyncFunction() {
    var m = monitor.start();
    setTimeout(function() { m.end(); }, Math.floor(Math.random()*1000));
}
for (var i = 0; i < 1000; i++) { asyncFunction(); }
process.on('exit', function() {
    console.log('Median runtime (ms): ' + monitor.stats['runtime'].percentile(0.5));
});

will output "Median runtime (ms): 497".

reporting

The reporting module provides a way to graph values over time and present it in a auto-updating HTML page. See test/reporting.test.js for examples or read the reporting module documentation.

var reporting = require('nodeload/reporting'), 
    stats = require('nodeload/stats'),
    report = reporting.REPORT_MANAGER.addReport('Random Numbers'),
    chart = report.getChart('Gaussian / Pareto vs. Time (minutes)');
for (var timeout = 0; timeout < 5000; timeout+=500) {
    setTimeout(function() {
            chart.put({
                'Pareto': stats.nextPareto(0, 100),
                'Gaussian': stats.nextGaussian()
            });
        }, timeout);
}

will display a graph on http://localhost:8000/ and save it to an HTML file in the local directory.

loop

The loop module provides a way to execute a function at a set rate and concurrency. See test/loop.test.js for examples or read the loop module documentation for details.

var http = require('http'),
    loop = require('nodeload/loop'),
    requests = 0,
    client = http.createClient(80, 'www.google.com'),
    l = new loop.MultiLoop({
        fun: function(finished) { 
            client.request('GET', '/').end();
            requests++;
            finished();
        },
        rps: 10,
        duration: 3,
        concurrency: 5
    }).start();
l.on('end', function() { console.log('Total requests: ' + requests) });

will output "Total requests: 30".

http

The http module provides a generic HTTP server that serves static files and that can be configured with new routes. See test/http.test.js for examples or read the http module documentation.

var http = require('nodeload/http');
var server = new http.HttpServer().start(10000);
server.addRoute('^/hello$', function(url, req, res) {
    res.writeHead(200);
    res.end("Hello");
});

http://localhost:8000/package.json will output the contents of ./package.json, and http://localhost:10000/resource will display "Hello".

CONTRIBUTING

File bugs on github, email any of the authors, and fork away. doc/developers.md has brief instructions on getting tests up and running, and will hold more design details in the future. Contributions are always welcome.

nodeload's People

Contributors

benschmaus avatar ctavan avatar erkan-yilmaz avatar jonjlee avatar orlandov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nodeload's Issues

Object #<Client> has no method 'destroy'

Hi,

When i run the quick start sample https://github.com/benschmaus/nodeload/blob/master/doc/nodeload.md, i have the next error :

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
TypeError: Object #<Client> has no method 'destroy'
    at /var/local/webroot/bookmarks-node/node_modules/nodeload/nodeload.js:9:506
    at Client.<anonymous> (/var/local/webroot/bookmarks-node/node_modules/nodeload/nodeload.js:10:180)
    at Client.emit (events.js:88:20)
    at ClientRequest.<anonymous> (http.js:1511:10)
    at ClientRequest.emit (events.js:67:17)
    at Socket.<anonymous> (http.js:1114:11)
    at Socket.emit (events.js:67:17)
    at Array.0 (net.js:301:25)
    at EventEmitter._tickCallback (node.js:192:40)

I use node v0.6.5

Thanks in advance

nested requests

Hi, am trying to perform a test of a webserver using requestLoop.

The flow is like this:

Threre is a MAIN_REQUEST:
* If success: I need to wait some time and trigger a SECONDARY request and then if success then I can consider it as SUCCESS

  • If it fails then it is an error.

I was trying to achieve that by using the following function it the requestLoop:

function check_request(finished, client, url, index) {
    url = url + get_dev(index);

    var req = client.request('GET', url, { 'host': 'localhost', 'port': 8383 });
    req.on('response', function(res) {
        var body = '';
        var chunks = 0;
        res.setEncoding('utf8');
        res.on('data', function(data) {
            body = body + data.toString();
        });

        res.on('end', function() {
            if (body.indexOf("wait_retry") ==-1) {
                arr[index] = {status: 'BOOTING', wait: getRandomInt(0,max_boot_time)};
                //setTimeout(do_ack, arr[index].wait, client, index, finished, req, res);
                setTimeout(function() {
                    console.log("HOLA");
                    finished({req: req, res: res});
                }, 5000);
            } else {
                var sleep_regex = /sleep ([0-9]+)/m
                var sleep_array = body.match(sleep_regex);
                arr[index] = {status: 'WAITING', wait: parseInt(sleep_array[1],10)};
                res.statusCode = 301;
                finished({req: req, res: res});
                //setTimeout(check_request, arr[index].wait, finished, client, 'request', index, req);
            }
            console.log(url + " ******* " + arr[index].status);
            //finished({req: req, res: res});
        });
}

It seems that no new request is launched until finished is called. Is there a way to make the requestLoop call async?, so requests are launched no matter if finieshed was called or not.

Regards

nl.js doesn't support query strings?

When I use a command like this

nl.js -c 100 -n 4000 -i 1 'http://127.0.0.1:3000/api?q=test'

Nodeload does not use the query string in the resulting test (verified by server logs).
Is this expected behavior?

Colon in file name fails on Windows

Failure

npm install nodeload fails with npm ERR! Error: ENOENT, open 'C:\Users\Hans\AppData\Local\Temp\npm-1340271001668\1340271001668-0.7088631778024137\package\results-2012-02-01T05:32:46.827Z-stats .log'

Error

verbose unpack C:\Users\Hans\AppData\Local\Temp\npm-1340271990814\1340271990814-0.479529005009681\tmp.tgz failed to unpack the file results-2012-02-01T05:32:46.827Z-stats.log

Fault

Colons (:) in the name of the file results-2012-02-01T05:32:46.827Z-stats.log makes it invalid in NTFS.

suggest remote to support redis to synchronize request.

I'm very new to nodeload. I'm impressed with nodeload to perform single process benchmark of my application. Then, I want to do a clustered test. I read about the doc, which suggest me to have remote to work.

I gave it a try and found it's hard to splitting the benchmark requests which read from a single file line by line to the target nodes. Especially make them to synchronize. So I think a central redis database can be put to use, to serve benchmark request one after another, eg: use rpop to pop out the request from a list. Basically to make requestGenerator() to return request stored in a redis instance.

I managed to changed requestGenerator() to do it, but it requires redis to be imported on the slave nodes. Mind to point me where to add this redis import? Or, do you have any better idea to do the same?

Not compatible with Node 0.6

I am getting messages about being unsupported when installing with npm.

node 0.6.0
npm 1.0.105

package.json refers to node 0.4
requires.path is also removed in favour of node_modules

When is an update likely?

client.request in examples code breaks node.js

'client.request()' is used in five of the /examples files and breaks the current nodeJS (v0.8.9). Example:

requestGenerator: function(client) {
return client.request('GET', "/" + Math.floor(Math.random()*8000), { 'host': 'localhost' });
}

terminal output from simpletest.ex.js:

Test server on localhost:9000.
http.createClient is deprecated. Use http.request instead.
Started HTTP server on localhost:8000.

events.js:66
throw arguments[1]; // Unhandled 'error' event
^
Error: listen EADDRINUSE
at errnoException (net.js:769:11)
at Server._listen2 (net.js:909:14)
at listen (net.js:936:10)
at Server.listen (net.js:985:5)
at Object. (/home/joseph/node/load_test.js:15:4)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)

Changing 'client' to 'http' fixes the problem:

requestGenerator: function(http) {
return http.request('GET', "/" + Math.floor(Math.random()*8000), { 'host': 'localhost' });
}

Customizeble Test Result page

How can i customize Test Result page without modify module files(from my code)
For example add legend to Histogram ?

Integration exsiting node-http-proxy server with memcached library

Hi ,
I am new node.js , i wrote a node-http-proxy server and it works very well but it is not included cached feature to memcached any webpages that visited.
I know have any memcache library now such as : node-memcached, node-connect, node-static , ...

but i don't know how to migration them to my proxy server !

some body help me to do this work ?

thank !

socket hang up problem with nodejs 0.8.2

Hi

while using nodeload accross many servers (dstributed onto EC2 instances), I saw strange bandwidth collapsing although my scenario was a simple "ramp up".
After investigating, the collapsing was caused by some slaves who went into a strange state :

  • their cpu usage falls from 40% to almost 0%
  • they still send data to the master but values are zero...
  • in the logged output I see :
    ......WARN: Error during HTTP request: Error: socket hang up......... ................................
  • tcpdump -i eth0 -An port 80 shows me NOTHING, so the slave is no longer connecting

I think the socket reaches a timeout, but the reconnection code doesn't work, but I'm not a NodeJS expert .

here is my setup :

  • 1 master
  • 15 EC2 slaves running nodeload.js
  • the tested service is an amazon elastic load balancer with autoscaling group with apache/tomcat servers

the code that I use from the master :

var trk_test = {
        name: "30min ramp",
        host: targetHost,
        port: 80,
        timeLimit: 1800,
        loadProfile: [[0,0], [1800,500] ],
        numUsers: 1,
        stats: [/*..removed stuff..*/],
        requestGenerator: function(client) {
             /*..removed stuff..*/
            var request = client.request('GET', uri, headers);
            request.end();
            return request;
        }
};

console.log('\nStarting test.\n');
var cluster = new nl.LoadTestCluster('MYPUBLICIP:8601', slaveHosts);

// gracefully propagate the interruption to the slaves
process.on('SIGINT', function () {
        console.log('\nGot SIGINT. Stopping test\n');
        cluster.end();
});

cluster.run(trk_test);

cluster.on('end', function() {
        console.log('\nAll tests done. exiting.\n');
        process.exit(0);
});

Any idea of what happens ?

TargetRps

Is targetRps per user or overall?

Unable run my test file when I have increased the num of users

Hi
In my test file when I increase the num of users I was getting the warning that "WARN: Error during HTTP request: Error: ECONNREFUSED, Could not contact DNS servers" and Iam unable rectify the problem so please help me in solving this error.

nl.run({
name: "test",
host: 'http://91.123.195.57/',
port: 9200,
numUsers: 2000, //Increased my num of user
timeLimit: 180,
targetRps: 500,
stats: [
'result-codes',
{ name: 'latency', percentiles: [0.9, 0.99] },
'concurrency',
'rps',
'uniques',
{ name: 'http-errors', successCodes: [200,404], log: 'http-errors.log' }
],

HTTPS Support?

I was checking this out (very cool tool) for testing some HTTPS stuff were doing... I didn't see any explicit support for it mentioned anywhere, so I played around with the code and tried the quick and easy way of changing all the

require('http')

to

require('https')

However, connecting the clients to SSL seems not to work.. Has anyone done this or where else should I look?

Cheers

J

nodeload in npm repo is old

The version of npm that gets installed with 'npm install nodeload' is considerably different from the latest code. Kind of confusing, especially when first approaching this library. The new code is way better :)

Socket hang up error

Hi,
when i try to create my proxy server with connect-cache and connect middleware , i get an error :
^
Error: socket hang up
at createHangUpError (http.js:1092:15)
at Socket.onend (http.js:1155:27)
at TCP.onread (net.js:349:26)

the line create server has bellow content :
--->connect.createServer(connect_cache({rules: [{regex: /.*/, ttl: 60000}]}),function(b_request, b_response){ .....}

any body help me to fix error ?

thank !

Can not run node.js app with web client

0 down vote favorite

I wrote and ran a node.js program (such as : hello.js or others) on my linux server with ip address 62.x.x.x with basic content :

var http = require('http');
http.createServer(function(req,res){
res.writeHead(200, {'content-type':'Text plain'});
res.end('Hello ');
}).listen(8000)

i tried test it on server with curl command : curl http://127.0.0.1:8000

i got result : 'hello' on screen.

but when i tried it on my client machince with webbrowser client (IE, Firefox,...) http://62.x.x.x:8000
The webbrowser can not load this page and can not return my resutl .

I don't know what dose error mean ? , so some body can show me ? and how to fix error ?

thank !

HTTP_PORT environment variable needs clearer documentation

nodeload needs listen on a specific http port, the default is 8000, which can be configured with HTTP_PORT environment. I happened have another server listened on 8000, it makes nodeload unable to start, showing ADDRESS ALREADY IN USE.

It getting me look at the source code to figure out what's going wrong. I think this HTTP_PORT should be highlighted in the documentation.

some parts of the manual are confusing

hi,

perhaps I didn't understand this part of the doc (nodeload.md) ?

                                        // Specify one of:
numUsers: 10,                           //   1. numUsers: number of virtual users concurrently
                                        //      executing therequest loop
loadProfile: undefined,                 //   2. loadProfile: array with requests/sec over time:
                                        //        [[time (seconds), rps], [time 2, rps], ...]
                                        //      For example, ramp up from 100 to 500 rps and then
                                        //      down to 0 over 20 seconds:
                                        //        [[0, 100], [10, 500], [20, 0]]

                                        // Specify one of:
targetRps: Infinity,                    //   1. targetRps: times per second to execute request loop
userProfile: undefined,                 //   2. userProfile: array with number of users over time:
                                        //        [[time (seconds), # users], [time 2, users], ...]
                                        //      For example, ramp up from 0 to 100 users and back
                                        //      down to 0 over 20 seconds:
                                        //        [[0, 0], [10, 100], [20, 0]]
  • I must specify numUsers OR loadProfile which are related to 2 different things !?
    (one setting is about number of users, the other about RPS) ?!?
  • I must specify targetRps OR userProfile :
    same remark

Perhaps it is a typo ? IMHO you wanted to write

  • specify one of "numUsers" or "userProfile"
  • specify one of "targetRps" or "loadProfile"

Am I wrong ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.