nosyndicate/pytorchrl

Deep Reinforcement Learning algorithms implemented in PyTorch

PyTorchRL

Deep Reinforcement Learning implemented in pytorch.

This project trying to mimic the structure of rllab code and occasionally borrow code from rllab and inverse_rl

• Implemented Algorithms

  • Online

    • DDPG (Deep Deterministic Policy Gradient)
    • NAF (Normalized Advantage Function)
    • Soft Q-Learning (Deep Energy-Based Policy)

  • Batch

    • TRPO (Trust Region Policy Optimization)

  • Inverse Reinforcement Learning & Imitation Learning

    • Generative Adversarial Imitation Learning
    • Adversarial Inverse Reinforcement Learning

pytorchrl's Issues

conjugate gradient code will raise error

Conjugate gradient code will complain divides by 0 error if using numpy 1.13.3 in environmental.yaml
on Mac OS on anaconda, so we have to use 1.13.1. But 1.13.1 will raise error on linux. yuck!

Migrating to use pytorch 0.40

It's now worth consider to migrate to pytorch 0.40 as there is several advanced changes.

Trying to get AIRL example to work

Hey.

I am trying to get the framework to work, however I fail compile most of the code.

Can you provide some install instructions?

Thank you!

TRPO code is not performing well compared to Other implementations

Pearlmutter method only gives "good" value of hessian vector product in the first two iterations in conjugate gradient loop

Speed issue of implementation

DDPG code is really slow on linux machine with 12 threads, the speed of training 1 epoch (10000 steps) is 56 seconds which is same as just use one thread on my mac. The cpu is Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz for my mac, and Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz for my desktop linux machine.

According to https://github.com/Khrylx/PyTorch-RL, solution seems to be export OMP_NUM_THREADS=1, this will limit to just use one thread on linux machine, which is about same speed as mac with just one thread.