spirals-team/hadoop-benchmark

Docker containers to build an Hadoop infrastructure and experiment feedback control loops atop of it.

hadoop-benchmark's Issues

Add docker registry to the controller so images are built only ones

1. Add to the create cluster:

start_container $controller_node_name local-registry -d -p 5000:5000 -h local-registry registry

2. Update the build-image to push the result to the registry
3. Add the prefix for all images consul:5000/

[Dockerfile] Adopt Docker Best Practices

Can you carefully read and adopt the Docker best practices document to write your Dockerfile ?

MapReduce function memory values

Where can these values set in this system: yarn.scheduler.maximum-allocation-mb, mapreduce.map.memory.mb, mapreduce.map.java.opts, mapreduce.reduce.memory.mb, mapreduce.reduce.java.opts?
I haven't found containing XML files.

[DockerHub] Adding the support for automated build in GitHub

1. Use the Spirals-Team account on DockerHub (ask Maxime)
2. Use the automated build support for GitHub
3. Upload each benchmark container on DockerHub to leverage its deployment at scale

Docker installation

How can I install docker and docker-machine inside user folder on Grid5000?

Support Windows

Currently, the project has only been tested on Linux and OSX. It should also work on Windows. Once anyone has an access to a Windows machine, it would be great to give it a try.

Error creating machine: Error waiting for machine to be running: Maximum number of retries (60) exceeded

I am on Grid5000 and have a question, again.
I have got error message above for command: CONFIG=g5k_cluster ./cluster.sh create-cluster

I have already set IP addresses and added home to Path: /usr/local/bin:/usr/bin:/bin:/grid5000/code/bin:/home/kakos

How should I fix this?

generic driver does not support start

I have got following error during cluster creation process on 'sagittaire'.

[cluster.sh]: checking status of docker machine: g5k-hadoop-consul
[cluster.sh]: docker machine g5k-hadoop-consul is stopped, starting...
[cluster.sh]: ===> docker-machine start g5k-hadoop-consul
Starting "g5k-hadoop-consul"...
generic driver does not support start

It was working earlier.

Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded

There is a problem with Docker. It started successfully 2 virtual machines, and throw an error when creating third one.

Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded

I have eliminated all the other problems, but it appears maybe randomly.