zalando-incubator/docker-locust

Docker image for the Locust.io open source load testing tool

docker-locust's Issues

Using Python 3

Stats are randomly missing for distributed and single automated mode.

Locust has a reports collection and generation delay, that depends on the amount of data generated by slaves per second. In the meantime, delay of 3 seconds had been observed, but it can probably be higher for a larger test scenario.

Static delay has been introduced in #57

Reduce the image size

Reporting in XML format

Unable to set the log level and --reset-stats / --no-reset-stats

These args are hardcoded, but it might be desired to customize them, e.g.:

• log level may affect performance
• one may want to minimize console output for large tests
• for tests where the ramp up duration is comparable to the test duration, the RPS stats are noticeably different for different reset-stats mode

Controller process can be removed

In order to simplify app's architecture, the controller process could be removed in favor of --no-web and --expect-slaves options.

Improve the html report or remove the feature

Current formatting of html report is a disaster:
locust_report.html.zip

The proposal is to review the feature and either improve or remove it.

Improve README

Based on the Zalando OSS Readme Template, I think we can improve the following items:

• Expand requirements
  • Add docker version requirements
  • Clarify installation/usage/deployment process
• Make clear what version of locust is used in the current release
  • If needed, establish a release process (not really related to the README, could end up in a different issue).
• Develop a "How to" guide, a tutorial with examples to ease adoption.
• Add contributing guidelines (with instructions on how to build and test it)
• Include a SECURITY.md
• Add code quality

Trigger CDP build if there is a new ubuntu base image

Update locustio to 0.8.x

It'd be nice to update to 0.8.x branch as it contains some basic graphs in the UI.

bootstrap incorrectly reports that all slaves are connected

During bootstrap a controller node checks in app.py that the number of connected slaves is actually equal to desired number of slaves. This check, however, fails if the controller times out waiting for all slaves to come online. Instead, the controller - correctly - warns that the number of online slaves is not equal to the desired number of slaves, yet still proceeds:

5:34:47 PM INFO:bootstrap:Currently connected slaves: 0

5:34:52 PM INFO:bootstrap:Checking if all slave(s) are connected.

5:34:52 PM INFO:bootstrap:Currently connected slaves: 0

5:34:57 PM INFO:bootstrap:Checking if all slave(s) are connected.

5:34:57 PM INFO:bootstrap:Currently connected slaves: 0

5:35:02 PM WARNING:bootstrap:Connected slaves:0 != defined slaves:8

5:35:02 PM INFO:bootstrap:All slaves are succesfully connected! Start load test automatically for 150 seconds.

The issue is here:

docker-locust/src/app.py

Line 107 in 1f19a41

logger.warning('Connected slaves:{con} != defined slaves:{dfn}'.format(

Instead, app.py should bail out with an error.

Remove the "Open source with more useful features coming" from the Advantages section

Since docker-locust itself is not a tool, but rather a wrapper, it doesn't compete with the commercial load testing tools. Provided advantages should base on docker-locust comparison with Locust itself or the other Locust wrappers (e.g. https://github.com/hakobera/docker-locust)

Docker/Unix Idioms and Security.

Hi, thanks for putting this together. Just a few feedback items to consider:

• Eschew bash wrappers over docker and docker-compose (especially curl to bash ). Responsible users will have to read every line of the bash script before running it anyway. Which is a waste since everything this script does can already be done with plain old Docker or Compose. It would be better to just prepare a ready made docker-compose.yml file that can be copy/pasted and just run TARGET_HOST=example.com docker-compose up -d to run the entire stack.
• The image URL in the README is not hosted in Docker hub. This is concerning from a security POV since we now have to trust an unknown third party container registry to use it. Whereas on Docker Hub, we have community ratings, security scanning etc.
• Make it clear why AWS credentials are needed (-v ~/.aws:/root/.aws) to run a single instance of the stack. It currently doesn't seem required to run this locally yet the aws credentials are volume mounted in all the examples.

Integrate with CDP to release new docker image

Error after upgrade to python3

After #87 and #88 I started to get errors with the locally built image:

Traceback (most recent call last):
  File "/usr/local/bin/locust", line 7, in <module>
    from locust.main import main
  File "/usr/local/lib/python3.5/dist-packages/locust/main.py", line 14, in <module>
    from . import events, runners, web
  File "/usr/local/lib/python3.5/dist-packages/locust/runners.py", line 13, in <module>
    from gevent.pool import Group
  File "/usr/local/lib/python3.5/dist-packages/gevent/pool.py", line 27, in <module>
    from gevent._imap import IMap
  File "/usr/local/lib/python3.5/dist-packages/gevent/_imap.py", line 227, in <module>
    import_c_accel(globals(), 'gevent.__imap')
  File "/usr/local/lib/python3.5/dist-packages/gevent/_util.py", line 105, in import_c_accel
    mod = importlib.import_module(cname)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "src/gevent/_queue.pxd", line 18, in init gevent.__imap
ValueError: gevent._queue.Queue has the wrong size, try recompiling. Expected 80, got 72

The PRs have been temporarily reverted until the issue is found.

Ability to read multiple files from s3 and http/https

docker-locust can only read one single file from s3 and http/https resource. It should be able to read multiple files and choose which one is the load test script and which one is additional file. Sample case: 1 python file is load test script and 1 json file is file that store the payload.

Where can I verify that the Locust tasks were run against the Target URL?

Hey, thanks for this repo, all looks great and seems to be functioning well.

I have a html report, I can access the CSV files also. However, I cannot verify that the tests were run against my Target URL. Is there an indicator to state what the tests were run against? i.e - that they weren't just run against / or localhost on the same machine.

I set the URL initially, but afterwards there is no indicator I can see. When I echo out $TARGET_HOST, $TARGET_URL and `$TARGET' I'm getting a blank also.

Thanks.

How is this different from other Docker images for Locust.io?

What makes this different from any of these Docker images? Putting a few lines about this in the README will be helpful to anyone external who might be able to use it.

Controller process should check for the desired amount of slaves to come up

Currently in the AUTOMATIC mode the Controller waits for Master web UI to come up and then immediately starts the test.
In order to support distributed environments (e.g. AWS) where instance spin up time may take several minutes, it should also wait for all Slaves to be connected to the Master, until certain time limit is reached.

Document how to run distributed test and deploy in AWS

Distributed testing is an important Locust feature, which is also mentioned in project's Key advantages: "It is also possible to be deployed in AWS to create bigger load". However, there's no any mentioning of distributed flow or AWS deployment in project's README.

Fail with a reasonable error message if scenario file is not available

It should be possible to use the Docker image directly as `docker run` instead of `local.sh`

Similarly to https://github.com/hakobera/docker-locust
It would simplify the flow and allow to get rid of docker-compose.

Abort Jenkins based load test

When running locust as a Jenkins job (as described in the readme), aborting the Jenkins job does not actually abort the running load test.

All the spawned docker containers are still running after the job has been canceled and they still create load.

It would be great, if aborting the Jenkins job would also abort the running load test.

I see three approaches how this could work:

1. intercept the signal sent from jenkins for aborting and do some cleanup
2. provide a how-to for a post-build action that does the cleanup if necessary
3. also spawn a watchdog, that checks if the job is still running and that will cleanup (including itself) after the job has been cancelled

I think, 1. would be the best, and 2. probably the easiest

Shows all logs for automatic run mode

Container not building

Hi there, the issue I am having is that I build the container, which is pulling from FROM registry.opensource.zalan.do/stups/ubuntu:16.04-60

then run ./local.sh deploy

Sending build context to Docker daemon   382 kB
Step 1/12 : FROM registry.opensource.zalan.do/stups/ubuntu:16.04-60
 ---> e3d1f57f68e0
Step 2/12 : ENV REQUESTS_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
 ---> Using cache
 ---> 0c9f403ed6b1
Step 3/12 : RUN apt-get update && apt-get install -y python-dev python-zmq python-pip
 ---> Using cache
 ---> f205011bd69b
Step 4/12 : EXPOSE 8089
 ---> Using cache
 ---> 520046f518de
Step 5/12 : EXPOSE 5557
 ---> Using cache
 ---> 58deb910ad03
Step 6/12 : EXPOSE 5558
 ---> Using cache
 ---> f9dd9b1ddd33
Step 7/12 : COPY requirements.txt /tmp/
 ---> Using cache
 ---> 103daacc086b
Step 8/12 : RUN pip install -r /tmp/requirements.txt
 ---> Using cache
 ---> ee12e515f513
Step 9/12 : COPY . /opt/
 ---> b0e64588c9b1
Removing intermediate container 152600dbef69
Step 10/12 : WORKDIR /opt
 ---> 94af69a003f1
Removing intermediate container 3fc81a7bd97b
Step 11/12 : ENV PYTHONPATH .
 ---> Running in c451832efdb5
 ---> 6b28a22a0c7f
Removing intermediate container c451832efdb5
Step 12/12 : CMD /usr/bin/python src/start.py
 ---> Running in 548581acbb36
 ---> 9e47141bd1dd
Removing intermediate container 548581acbb36
Successfully built 9e47141bd1dd```

`root@hq-stage-docker06:/home/toor/workspace/qa-docker-locust-io# docker ps`

reveals no locust containers... it used to work...

now when I connect to http:www.localhost.com:8089 (black page)...

Machine-friendly reports are unavailable in the automatic run mode

The controller only downloads html report, however automatic mode usually assumes automatic parsing and values comparison. #57 enables Controller to download locust .csv and .json reports from a Master node.

Error for CI Command

I used the CI bash example command, I get the following error instead of any success.

Creating docker_locusts_controller ... done
Creating app_slave_1      ...
Creating app_slave_1      ... done
Locust application is successfully deployed. you can access http://<docker-host-ip-address>:8089
Error response from daemon: Could not find the file /opt/reports in container docker_locusts_controller

Move to Python 3

Travis CI build shows success on errors

https://travis-ci.org/zalando-incubator/docker-locust/builds/309572144?utm_source=github_status&utm_medium=notification

Document environment variables accepted by the container

Also, it would be nice to change the long lines to multiline code blocks in order to remove a need of horizontal scrolling and provide an ability to compare different run methods.