rl_algs

[openai]

rl_algs

[ViktorM]

@kvfrans the project and results are great. In addition it would be nice to have this repo tested and working out of the box at least for the most used platforms, like Ubuntu 16.04 Python 3.5+ or 3.6+. And/or more detail instructions for running the demos. Following current readme doesn't lead to running code without errors.

[TheCrazyT]

Yeah i believe ther is something wrong here.
I solved the problem by creating a symbolic link of rl-algs into mlsh_code with the name "rl_algs".
i then did:
touch rl-algs/__init__.py

Not to mention the other problems with latest mujoco that i solved by:
pip3 install mujoco-py==0.5.7
(using python 3 here)
and using "mjpro131 linux" from https://www.roboti.us/index.html.

as a dirty workaround I also did:
sudo cp -R envs/mujoco /usr/local/lib/python3.5/dist-packages/gym/envs/

oh well and i added the following on top of main.py:

from gym.envs.registration import registry, register, make, spec
register(
     id='AntBandits-v1',
     entry_point='gym.envs.mujoco:AntBanditsEnv',
     max_episode_steps=1000,
)

I guess as alternative you could also modify the /usr/local/lib/python3.5/dist-packages/gym/envs/__init__.py file according to the __init__.py

in the root directory of this repository.
But thats as dirty as the workaround above i guess.

If somebody knows a clean way of using the project it would be nice :)

PS:
Not shure if everything is ok now but i get atleast the following output:

It is iteration 0 so i'm changing the goal to [0 5]
1: global: -0.841579880105, local: -0.841579880105
2: global: -0.788466165552, local: -0.788466165552
3: global: -0.794958083416, local: -0.794958083416
4: global: -0.778563928617, local: -0.778563928617
5: global: -0.787106789342, local: -0.787106789342

I was expecting some animations like a saw in a youtube video :(
EDIT:
Oh well .. figured it out, the master.py misses env.render() ,so there is no direct option for it.
You could place it in some loop to see whats happening, but its quite slow.

EDIT AGAIN:
Uhm I was wrong there is actualy a "--replay True" param to display an animation. (with much better framerate).
For some reason i did not check the rollouts.py
But be careful with that, it won't create checkpoints if you use it.

[gautam1858]

thank you but when I run the code it is giving me this error, could you please help me to solve this error as I am unable to locate ant-bandits in gym env

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 105, in <module>
    main()
  File "main.py", line 102, in main
    train()
  File "main.py", line 95, in train
    master.start(callback, args=args, workerseed=workerseed, rank=rank, comm=comm)
  File "/Users/neutrino/mujoco/mlsh/mlsh_code/master.py", line 15, in start
    env = gym.make(args.task)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/gym/envs/registration.py", line 161, in make
    return registry.make(id)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/gym/envs/registration.py", line 118, in make
    spec = self.spec(id)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/gym/envs/registration.py", line 147, in spec
    raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: antBandits-v1

[kvfrans]

Hi, I've pushed an update that should contain the relevant classes from rl-algs

[gautam1858]

unable to find tinkerbell package, please point out a resource and It would be helpful if you add requirement.txt file, thank you :)

[kvfrans]

cleaned out the references to tinkerbell, let me know if any libraries still missing

[gautam1858]

rl-algs distributions.py is missing, the file is called in policy_network.py line 4, please add that file as well

[kvfrans]

added

[ViktorM]

Hi, have similar error with rl_algos:

mlsh/mlsh_code$ python main.py --task AntBandits-v1 --num_subs 2 --macro_duration 1000 --num_rollouts 2000 --warmup_time 20 --train_time 30 --replay False AntAgent
Traceback (most recent call last):
  File "main.py", line 20, in <module>
    from rl_algs.common import set_global_seeds, tf_util as U
ModuleNotFoundError: No module named 'rl_algs'

[kvfrans]

Hey, the structure of the repo has been done so it should be easier to install everything. # #

[ViktorM]

Thanks Kevin! Now it looks much more clear for me, will try as soon as get to my PC.

[JordanYeomans]

For anyone still experiencing this issue, I had to install mpi4py

pip install mpi4py

[Muguangfeng]

Hi, How do you run the code? How long did the training take? I trained the task'Ant Obstacles Gen-v1'. It is so slow. The other tasks are also very slow. Is there any solution?

[PraveenVenugopal]

Facing this issue when i run main.py

Traceback (most recent call last):
File "/home/venugopal/anaconda3/envs/mlsh/lib/python3.5/site-packages/gym/envs/registration.py", line 132, in spec
return self.env_specs[id]
KeyError: 'AntBandits-v1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "../mlsh_code/main.py", line 87, in
main()
File "../mlsh_code/main.py", line 84, in main
train()
File "../mlsh_code/main.py", line 77, in train
master.start(callback, args=args, workerseed=workerseed, rank=rank, comm=comm)
File "/home/venugopal/mlsh/mlsh_code/master.py", line 15, in start
env = gym.make(args.task)
File "/home/venugopal/anaconda3/envs/mlsh/lib/python3.5/site-packages/gym/envs/registration.py", line 156, in make
return registry.make(id, **kwargs)
File "/home/venugopal/anaconda3/envs/mlsh/lib/python3.5/site-packages/gym/envs/registration.py", line 100, in make
spec = self.spec(path)
File "/home/venugopal/anaconda3/envs/mlsh/lib/python3.5/site-packages/gym/envs/registration.py", line 142, in spec
raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: AntBandits-v1

How to solve this error ??