papoudakis/asynchronous_rl

asynchronous_rl's Issues

Breakout performance - timesteps/time?

Hi,

I have been trying to train your a3c model on breakout game.

I am running it on a p2.xlarge instance and the params I am using are:

tmax: 40m
num_concurrent: 16
everything else is default

I have changed the network from a NIPS network to a NATURE network (has to give better performance).

I have a few questions that I wanted to ask! :)

• So far, the model has been running for 4m global timesteps (self.T) /300k actor_learner_thread timesteps (counter). In your results, you have posted your breakout results after "10m" timesteps - which timestep are you referring to? is the local one or the global one?

• I am training on a NVIDIA K80 (p2.xl instance), I am only able to run about 330k global time steps (self.T) in 1 hr - isn't that a bit slow? any comments on this?

• Do you have any saved plot/tensorboard data for training/testing you have done?

Thank you!