tdb-alcorn/q2

A framework and command line interface for reinforcement learning development

q2

A reinforcement learning framework and command line tool.

Get it

pip install q2
q2 --help

Use it

Make a folder, cd to it, then run

q2 init

to generate a new reinforcement learning project.

Docs

Documentation is available at https://q2.readthedocs.io/.

Develop

0. You'll need Python >= 3.5 (this is because q2 makes use of type annotations)
1. Clone this repository
2. I recommend setting up a virtual env in which to develop: python -m venv env
3. pip install -r requirements.txt

To test out the command line tools from your working copy of q2, run

pip install -e .

then run

q2 --help

to test that it worked.

Contributing

If you want to submit a bug fix or minor change, feel free to make a pull request.

If you want to discuss bigger improvements, send an email to tdbalcorn at gmail dot com.

q2's Issues

AttributeError: 'Namespace' object has no attribute 'out_filename'

Hi,

Trying to reproduce the cart pole training scenario described in here, I encountered the following error:

(python3)  ~/repos_and_code/reinforced_test ~> q2 train random --env gym.CartPole-v1 --episodes 10 --render
Traceback (most recent call last):
  File "/home/guillermo/.anaconda/envs/python3/bin/q2", line 11, in <module>
    sys.exit(main())
  File "/home/guillermo/.anaconda/envs/python3/lib/python3.6/site-packages/q2/tool.py", line 83, in main
    args.func(args)
  File "/home/guillermo/.anaconda/envs/python3/lib/python3.6/site-packages/q2/train/__init__.py", line 90, in main
    regimen.train(env_maker, state, args.num_epochs, args.episodes_per_epoch, test=args.test, render=args.render, bk2dir=args.bk2dir, out_filename=args.out_filename)
AttributeError: 'Namespace' object has no attribute 'out_filename'

Looking at the code, seems like the CLI argument out_filename is not defined anywhere.

More info:

Python version: Python 3.6.5 :: Anaconda, Inc.

Reproduce: Just follow the steps described here on a clean python environment.

Paths in objects.yaml should be relative to root of repo, not absolute

Absolute paths break collaboration across different machines

Running random agent from tutorial is broken

╰─$ q2 train random --env gym.CartPole-v1 --episodes 10 --render                               2 ↵
Traceback (most recent call last):
  File "/Users/tom/dev/team-halite/env/bin/q2", line 8, in <module>
    sys.exit(main())
  File "/Users/tom/dev/team-halite/env/lib/python3.6/site-packages/q2/tool.py", line 83, in main
    args.func(args)
  File "/Users/tom/dev/team-halite/env/lib/python3.6/site-packages/q2/train/__init__.py", line 38, in main
    from q2.agents import all_agents
  File "/Users/tom/dev/team-halite/env/lib/python3.6/site-packages/q2/agents/__init__.py", line 9, in <module>
    from .agent import Agent
  File "/Users/tom/dev/team-halite/env/lib/python3.6/site-packages/q2/agents/agent.py", line 7, in <module>
    class Agent(ABC):
  File "/Users/tom/dev/team-halite/env/lib/python3.6/site-packages/q2/agents/agent.py", line 21, in Agent
    sess:tf.Session,
AttributeError: module 'tensorflow' has no attribute 'Session'