soumith/deepmind-atari

deepmind-atari's Issues

Training with multiple GPU

Is it possible to train the DQN using multiple GPUs?

Segfault when running ./run_gpu and ./run_cpu

I followed the instructions in README.md after creating a new user account atari due to some version issues with my pre-existing installation of torch.

After setting ulimit -c unlimited and running gdb ../torch/bin/luajit core, I get the following clue as to what is causing the segfault.

Core was generated by `../torch/bin/luajit train_agent.lua -framework alewrap -game_path /home/atari/d'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fab3edf1d64 in ale::StellaEnvironment::emulate(ale::Action, ale::Action, unsigned long) () from /home/atari/deepmind-atari/torch/lib/libxitari.so

I'd like to continue diving into the problem, but I could use some help on how to proceed. Should I be raising this issue with https://github.com/deepmind/xitari instead of here since xitari seems to be behind my issue?

What is the role of xitari module ?

What is the role of xitari module ?

May be it is a dependancy for ale wrap module...

Thank you.

Moving from SpatialConvolutionCUDA to SpatialConvolution possibly broke things?

Hi!

Thank you for your fork of this project, and thank you for all your brilliant work on Torch.

I cloned this repository and ran it on one of my Titan Z GPUs for two to three days straight. It's been learning to play Breakout. However, when I use the resulting network file, enable graphics, and use qlua to display visual output, the agent seems to play no more intelligently than an untrained version -- it never scores more than a few points in a row. Here are the steps I took (source):

1. Cloned deepmind-atari.
2. Changed ../torch/bin/luajit train_agent.lua $args to ../torch/bin/qlua train_agent.lua $args in run_gpu.
3. Changed display=false to display=true in torch/share/lua/5.1/alewrap/AleEnv.lua
4. Ran ./run_gpu breakout for two to three days. This produced the network file dqn/DQN3_0_1_breakout_FULL_Y.t7.
5. Changed netfile="\"convnet_atari3\"" in run_gpu to netfile="\"DQN3_0_1_breakout_FULL_Y.t7\"".

My machine reached approximately 12 million steps. Maybe this is not enough?

I'm concerned that change 88fffea may have broken this demo. Is this at all possible? Or do you have any suggestions on what might be wrong? I greatly appreciate any input!

Running the code gives the following error

Running issue

When I tried to run a code I've got:

-framework alewrap -game_path /home/cave/atari22/deepmind-atari/roms/ -name DQN3_0_1_breakout_FULL_Y -env breakout -env_params useRGB=true -agent NeuralQLearner -agent_params lr=0.00025,ep=1,ep_end=0.1,ep_endt=replay_memory,discount=0.99,hist_len=4,learn_start=50000,replay_memory=1000000,update_freq=4,n_replay=1,network="convnet_atari3",preproc="net_downsample_2x_full_y",state_dim=7056,minibatch_size=32,rescale_r=1,ncols=1,bufferSize=512,valid_size=500,target_q=10000,clip_delta=1,min_reward=-1,max_reward=1 -steps 50000000 -eval_freq 250000 -eval_steps 125000 -prog_freq 5000 -save_freq 125000 -actrep 4 -gpu -1 -random_starts 30 -pool_frms type="max",size=2 -seed 1 -threads 4
../torch/bin/luajit: ./initenv.lua:8: module 'torch' not found:
    no field package.preload['torch']
    no file '/root/.luarocks/share/lua/5.1/torch.lua'
    no file '/root/.luarocks/share/lua/5.1/torch/init.lua'
    no file '/root/torch/install/share/lua/5.1/torch.lua'
    no file '/root/torch/install/share/lua/5.1/torch/init.lua'
    no file './torch.lua'
    no file '/root/torch/install/share/luajit-2.1.0-beta1/torch.lua'
    no file '/usr/local/share/lua/5.1/torch.lua'
    no file '/usr/local/share/lua/5.1/torch/init.lua'
    no file '/root/torch/install/lib/torch.so'
    no file '/root/.luarocks/lib/lua/5.1/torch.so'
    no file '/root/torch/install/lib/lua/5.1/torch.so'
    no file './torch.so'
    no file '/usr/local/lib/lua/5.1/torch.so'
    no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
    [C]: in function 'require'
    ./initenv.lua:8: in main chunk
    [C]: in function 'require'
    train_agent.lua:8: in main chunk
    [C]: at 0x00406260