onetimepad/advantage

A framework for making RL easy!

advantage's Issues

DQN train_iteration fix

The train_iteration method for DeepQModel has two if statements one for improve_policy_modulo and improve_target_modulo. If policy improvement happens, target improvement maybe shouldn't happen after ? (like one is a multiple of the other)

Approximators: ability to select variables to train

both for train, and restore for transfer learning

Learning Model Runner

Tests and complete learning model interface

Model/Agent/Approximator saving params and restoration

Model, Agent and Approximator need to have functionality for saving and restoring parameters.

Fix tests for approximators

has no attribute optimizer issue

OOP paradigm: separate uses of approximators_builder and base_approximators

The "config" parameter in base_approximators seems to bind approximators_builder and base_approximators together when the config should only be handled by approximators_builder. The proper pattern is used in agents_builder and base_agents. There might need to be separate "builder" for each approximator similar to agents. Basically, the config param shouldn't be passed around.

Approximators inference

There can be multiple feed_dict elements to for an approximator but only one concatenated input tensor. Have a way to keep track of all of them. Change the way inference() works.

Scripts for creating templates for adding new models

Pretty much a script that generates the models, agents, and protobufs files. Also always users to add elements, buffers, utils, etc... basically a CLI.

Wrapper in DiscreteActionSpaceAgent awkward

There is a wrapper (_action_wrapper) in DiscreteActionSpaceAgent. This wrapper is used to extract the one element np.array returned by the DQNAgent or any Discrete and ActionValue agent. However, it being placed in DiscreteActionSpaceAgent is a bit awkward, since technically an ActionValue agent could be continuous and a DiscreteAgent doesn't necessarily return an action that requires such a wrapper.

There should be a better fix for this...maybe a DiscreteActionValueAgent ?

See base_agents

Clean up commented out code

Test for DeepApproximators initialize method

Make DQNAgent tests deterministic

Currently the DQNAgent tests don't mock anything...

Configuration for Approximator Inputs

This would allow for more fine-grain control over what inputs go into an Approximator and allow for connection of multiple approximators together reducing the need for writing specific classes that deal with combined approximators.

See approximators.proto and inputs_placeholder in set_up()

Add scripts for running all tests

Sarsa needs to account for dimensions of elements

Specific Exceptions

Look for TODO's involving making exceptions more specific.

For example: "BadActionExcpetion" or something.

Split approximators into separate files