Problem in running simulation examples

[icarus-sim]

Problem in running simulation examples

[lorenzosaino]

@nboubakr thanks for the reporting this issue. I cannot reproduce the issue with my setup. How did you install Icarus? Did you install it with make install or did you do something else?

[nboubakr]

I am using Ubuntu 16.04 LTS, I cloned the git repository v0.7.0 and used make install.

[nboubakr]

The newest output errors:

boubakr@ubuntu:~/Icarus/icarus-v0.7.0$ make install
pip install --upgrade pip setuptools
Requirement already up-to-date: pip in /home/boubakr/.local/lib/python2.7/site-packages (18.1)
Requirement already up-to-date: setuptools in /home/boubakr/.local/lib/python2.7/site-packages (40.6.2)
pip install --upgrade -r requirements.txt
Requirement already up-to-date: pytest in /home/boubakr/.local/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (4.0.1)
Requirement already up-to-date: numpydoc in /home/boubakr/.local/lib/python2.7/site-packages (from -r requirements.txt (line 5)) (0.8.0)
Requirement already up-to-date: sphinx in /home/boubakr/.local/lib/python2.7/site-packages (from -r requirements.txt (line 6)) (1.8.2)
Requirement already satisfied, skipping upgrade: atomicwrites>=1.0 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (1.2.1)
Requirement already satisfied, skipping upgrade: py>=1.5.0 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (1.7.0)
Requirement already satisfied, skipping upgrade: pluggy>=0.7 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (0.8.0)
Requirement already satisfied, skipping upgrade: setuptools in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (40.6.2)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (1.11.0)
Requirement already satisfied, skipping upgrade: pathlib2>=2.2.0; python_version < "3.6" in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (2.3.3)
Requirement already satisfied, skipping upgrade: more-itertools>=4.0.0 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (4.3.0)
Requirement already satisfied, skipping upgrade: attrs>=17.4.0 in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (18.2.0)
Requirement already satisfied, skipping upgrade: funcsigs; python_version < "3.0" in /home/boubakr/.local/lib/python2.7/site-packages (from pytest->-r requirements.txt (line 2)) (1.0.2)
Requirement already satisfied, skipping upgrade: Jinja2>=2.3 in /home/boubakr/.local/lib/python2.7/site-packages (from numpydoc->-r requirements.txt (line 5)) (2.10)
Requirement already satisfied, skipping upgrade: alabaster<0.8,>=0.7 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (0.7.12)
Requirement already satisfied, skipping upgrade: snowballstemmer>=1.1 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (1.2.1)
Requirement already satisfied, skipping upgrade: packaging in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (18.0)
Requirement already satisfied, skipping upgrade: babel!=2.0,>=1.3 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (2.6.0)
Requirement already satisfied, skipping upgrade: imagesize in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (1.1.0)
Requirement already satisfied, skipping upgrade: Pygments>=2.0 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (2.3.0)
Requirement already satisfied, skipping upgrade: typing; python_version < "3.5" in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (3.6.6)
Requirement already satisfied, skipping upgrade: docutils>=0.11 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (0.14)
Requirement already satisfied, skipping upgrade: requests>=2.0.0 in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (2.20.1)
Requirement already satisfied, skipping upgrade: sphinxcontrib-websupport in /home/boubakr/.local/lib/python2.7/site-packages (from sphinx->-r requirements.txt (line 6)) (1.1.0)
Requirement already satisfied, skipping upgrade: scandir; python_version < "3.5" in /home/boubakr/.local/lib/python2.7/site-packages (from pathlib2>=2.2.0; python_version < "3.6"->pytest->-r requirements.txt (line 2)) (1.9.0)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /home/boubakr/.local/lib/python2.7/site-packages (from Jinja2>=2.3->numpydoc->-r requirements.txt (line 5)) (1.1.0)
Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /home/boubakr/.local/lib/python2.7/site-packages (from packaging->sphinx->-r requirements.txt (line 6)) (2.3.0)
Requirement already satisfied, skipping upgrade: pytz>=0a in /home/boubakr/.local/lib/python2.7/site-packages (from babel!=2.0,>=1.3->sphinx->-r requirements.txt (line 6)) (2018.7)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/boubakr/.local/lib/python2.7/site-packages (from requests>=2.0.0->sphinx->-r requirements.txt (line 6)) (1.24.1)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/boubakr/.local/lib/python2.7/site-packages (from requests>=2.0.0->sphinx->-r requirements.txt (line 6)) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.8,>=2.5 in /home/boubakr/.local/lib/python2.7/site-packages (from requests>=2.0.0->sphinx->-r requirements.txt (line 6)) (2.7)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/boubakr/.local/lib/python2.7/site-packages (from requests>=2.0.0->sphinx->-r requirements.txt (line 6)) (2018.11.29)
pip install --upgrade -e .
Obtaining file:///home/boubakr/Icarus/icarus-v0.7.0
Requirement already satisfied, skipping upgrade: networkx>=1.6 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (2.2)
Requirement already satisfied, skipping upgrade: numpy>=1.4 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (1.15.4)
Requirement already satisfied, skipping upgrade: scipy>=0.16 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (1.1.0)
Requirement already satisfied, skipping upgrade: fnss>=0.6.1 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (0.8.2)
Requirement already satisfied, skipping upgrade: matplotlib>=1.5.3 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (2.2.3)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.3 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (2.7.5)
Requirement already satisfied, skipping upgrade: click>=6.6 in /home/boubakr/.local/lib/python2.7/site-packages (from icarus==0.7.0) (7.0)
Requirement already satisfied, skipping upgrade: decorator>=4.3.0 in /home/boubakr/.local/lib/python2.7/site-packages (from networkx>=1.6->icarus==0.7.0) (4.3.0)
Requirement already satisfied, skipping upgrade: mako>=0.4 in /home/boubakr/.local/lib/python2.7/site-packages (from fnss>=0.6.1->icarus==0.7.0) (1.0.7)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (2.3.0)
Requirement already satisfied, skipping upgrade: backports.functools-lru-cache in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (1.5)
Requirement already satisfied, skipping upgrade: subprocess32 in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (3.5.3)
Requirement already satisfied, skipping upgrade: pytz in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (2018.7)
Requirement already satisfied, skipping upgrade: six>=1.10 in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (1.11.0)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (1.0.1)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /home/boubakr/.local/lib/python2.7/site-packages (from matplotlib>=1.5.3->icarus==0.7.0) (0.10.0)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.9.2 in /home/boubakr/.local/lib/python2.7/site-packages (from mako>=0.4->fnss>=0.6.1->icarus==0.7.0) (1.1.0)
Requirement already satisfied, skipping upgrade: setuptools in /home/boubakr/.local/lib/python2.7/site-packages (from kiwisolver>=1.0.1->matplotlib>=1.5.3->icarus==0.7.0) (40.6.2)
Installing collected packages: icarus
  Running setup.py develop for icarus
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/home/boubakr/Icarus/icarus-v0.7.0/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps:
    running develop
    error: can't create or remove files in install directory

    The following error occurred while trying to add or remove files in the
    installation directory:

        [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-6745.write-test'

    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:

        /usr/local/lib/python2.7/dist-packages/

    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.

    For information on other options, you may wish to consult the
    documentation at:

      https://setuptools.readthedocs.io/en/latest/easy_install.html

    Please make the appropriate changes for your system and try again.


    ----------------------------------------
Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/home/boubakr/Icarus/icarus-v0.7.0/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps" failed with error code 1 in /home/boubakr/Icarus/icarus-v0.7.0/
Makefile:11: recipe for target 'install' failed
make: *** [install] Error 1

[lorenzosaino]

This is the issue:

    The following error occurred while trying to add or remove files in the
    installation directory:

        [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-6745.write-test'

    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:

        /usr/local/lib/python2.7/dist-packages/

    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.

You don't have permissions to write to /usr/local/lib/python2.7/dist-packages/. Try running sudo make install instead. That should fix it.

[nboubakr]

@lorenzosaino Thank you for your reply.
Unfortunately, I tried the command as superuser and got the following error:

boubakr@ubuntu:~/Icarus/icarus-v0.7.0$ sudo make install
[sudo] password for boubakr: 
pip install --upgrade pip setuptools
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main
Makefile:11: recipe for target 'install' failed
make: *** [install] Error 1

[lorenzosaino]

What if you run sudo -E make install?

[nboubakr]

The same error.

[lorenzosaino]

Alright, let me look into it and get back to you.

[lorenzosaino]

@nboubakr I looked into this and I reached the conclusion that this is an issue with your environment and not with Icarus.

If you pull the latest commits from the master branch you'll find a Dockerfile that will allow you to build Docker image to run in a container with a clean setup. To do so, you will need to install Docker on your machine, move to the Icarus repo and run:

docker build -t icarus .
docker run --rm -it icarus

This will create a Docker image, launch it in a container and give you shell access to it. From that shell, you can run commands to be executed on that container. You can now run the examples and everything works fine.

root@ac24b0559959:/icarus# cd examples/lce-vs-probcache/
root@ac24b0559959:/icarus/examples/lce-vs-probcache# icarus run --results results.pickle config.py
[11:07:04 2018-12-08|WARNING|main] RESULTS_FORMAT setting not specified. Set to PICKLE
[11:07:04 2018-12-08|WARNING|main] LOG_LEVEL setting not specified. Set to INFO
[11:07:04 2018-12-08|INFO|main] Launching orchestrator
[11:07:04 2018-12-08|INFO|orchestration] Starting simulations: 2 experiments, 1 process(es)
[11:07:04 2018-12-08|INFO|runner-MainProcess] Experiment 1/2 | Preparing scenario: Strategy: LCE
[11:07:04 2018-12-08|INFO|runner-MainProcess] Experiment 1/2 | Start simulation
[11:07:55 2018-12-08|INFO|runner-MainProcess] Experiment 1/2 | End simulation | Duration 51s.
[11:07:55 2018-12-08|INFO|runner-MainProcess] Experiment 2/2 | Preparing scenario: Strategy: PROB_CACHE
[11:07:55 2018-12-08|INFO|runner-MainProcess] Experiment 2/2 | Start simulation
[11:08:37 2018-12-08|INFO|runner-MainProcess] Experiment 2/2 | End simulation | Duration 42s.
[11:08:37 2018-12-08|INFO|orchestration] END | Planned: 2, Completed: 2, Succeeded: 2, Failed: 0
[11:08:37 2018-12-08|INFO|main] Orchestrator finished
[11:08:37 2018-12-08|INFO|main] Saved results to file /icarus/examples/lce-vs-probcache/results.pickle

To address your problem, you could use the container to run your experiments, which could be a good idea anyway because it provides you portable and reproducible environment that will always work no matter what changes you make to the Python installation on your machine. Otherwise, you may want to investigate the pip issue on your machine. Doing some Googling I found similar issues to yours, like pypa/pip#5240.

As a side note, if you are using your machine for working on other Python projects in addition to Icarus and you decide not to use Docker containers for your development work, you may want to consider using virtualenv and virtualenvwrapper, which allow you to create separate Python environments for each project, so if you make changes to the installed Python packages for one project it doesn't impact others. You can read more here: https://virtualenvwrapper.readthedocs.io/en/latest/

Hope this help!

[nboubakr]

@lorenzosaino Thank you so much.