wizdom13/rnd-pytorch

Random Network Distillation(RND) algo in Pytorch

RND-Pytoch

Random Network Distillation(RND) algo in pytorch

This is a PyTorch implementation of Exploration by Random Network Distillation paper.

Requirements

• python3.6
• PyTorch
• gym
• numpy
• PIL
• argparse
• tensorboardX

Usage:

Prepare

CUDA_VISIBLE_DEVICES=0

Training

python train.py

Enjoy

python enjoy.py

TensorboardX Graph

tensorboard --logdir runs

Open in browser http://localhost:6006

rnd-pytorch's Issues

train.py

Hello,

This is really nice pytorch edition. I want to do some research based on ur content. But when I run train.py, I met this problem. Do u have any ideas? Thank you so much!

GPU training: False
Sticky actions: True
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
ERROR: Couldn't open /anaconda3/envs/RND/lib/python3.6/site-packages/atari_py/atari_roms/montezuma_revenge.bin ...

a little confuse about epoch

in train.py file, you give epoch 3:
def train_model(args, device, output_size, model, rnd, optimizer, s_batch, target_ext_batch, target_int_batch, y_batch, adv_batch, next_obs_batch, old_action_probs):
epoch = 3

but in argument.py:
parser.add_argument('--epoch', type=int, default=4,
help='number of epochs (default: 4)')

so, which parameter do we use?

Thank you!

Zip ARgument #2 must support iteration

Hi, I'm getting a frustrating error on windows:

File "train.py", line 226, in main
    for parent_conn, action in zip(parent_conns, actions):
TypeError: zip argument #2 must support iteration

Num Workers is set to 1, everythin else is unchanged.

How long will get out of room7

Hello wisdom,

Thank you for your pretty neat code. We learnt a lot from it. One question we are not sure how long will model get out of room 7? We stayed at room 7 for about 1 days, current global step is 6M, rewards still keep at 2500. We found from 400 points to 2500 points is pretty fast. But at 2500, it did not make any progress.

tensorboardX 1.7 is bugged

You should change requirements and readme to show tensorboardX==1.6 is required since 1.7 seems to be bugged (read another issue on different repo saying it's broken) and train.py will not run with it