jingweiz/pytorch-dnc

Neural Turing Machine (NTM) & Differentiable Neural Computer (DNC) with pytorch & visdom

pytorch-dnc's Issues

Add Licensing

Hi, great work with this!
What licensing is this repo under?
Would be helpful to know so that I can ensure compatibility with my own personal projects!

bug in the passing of read_vec_vb

Latest PyTorch Issue

I know you mention not supporting python 3, but this looks like just an issue with pytorch.
I have built pytorch from HEAD to use the new features such as broadcasting.

➜  pytorch-dnc git:(master) python main.py 
Traceback (most recent call last):
  File "main.py", line 21, in <module>
    agent.fit_model()
  File "/home/jason/projects/pytorch-dnc/core/agents/sl_agent.py", line 115, in fit_model
    action = self._forward(self.experience.state1)
  File "/home/jason/projects/pytorch-dnc/core/agents/sl_agent.py", line 61, in _forward
    output_vb = self.circuit.forward(Variable(input_ts[i]))
  File "/home/jason/projects/pytorch-dnc/core/circuit.py", line 83, in forward
    self.read_vec_vb = self.accessor.forward(hidden_vb)
  File "/home/jason/projects/pytorch-dnc/core/accessors/static_accessor.py", line 48, in forward
    self.memory.memory_vb = self.write_heads.forward(hidden_vb, self.memory.memory_vb)
  File "/home/jason/projects/pytorch-dnc/core/heads/static_write_head.py", line 35, in forward
    super(StaticWriteHead, self).forward(hidden_vb, memory_vb)
  File "/home/jason/projects/pytorch-dnc/core/heads/static_head.py", line 104, in forward
    self._content_focus(memory_vb)
  File "/home/jason/projects/pytorch-dnc/core/heads/static_head.py", line 40, in _content_focus
    K_vb = batch_cosine_sim(self.key_vb, memory_vb)  # [batch_size x num_heads x mem_hei]
  File "/home/jason/projects/pytorch-dnc/utils/similarities.py", line 15, in batch_cosine_sim
    denominator = torch.sqrt(torch.bmm(u.norm(2, 2).pow(2) + epsilon, v.norm(2, 2).pow(2).transpose(1, 2) + epsilon))
  File "/home/jason/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py", line 706, in transpose
    return Transpose.apply(self, dim1, dim2)
  File "/home/jason/anaconda3/lib/python3.6/site-packages/torch/autograd/_functions/tensor.py", line 80, in forward
    result = i.transpose(dim1, dim2)
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

Support Python3

It should be possible to make it so that it works with python2 and python3.
Mainly just a bunch of print "..." --> print("...") changes usually.

Failed to establish a new connection: [Errno 111] Connection refused',)) Input:

+- - - - - - - - - - 1 1 1 1 1 1 1 1 - - - -+
+- - - - - - - - - - - - - - 1 1 1 1 - - - -+
+- - - - - - - - - - 1 1 1 1 1 1 1 1 - - - -+
+- - - - - - - - - - 1 1 1 1 1 - - - - - - -+
Exception in user code:

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/visdom/init.py", line 228, in _send
data=json.dumps(msg),
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 110, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 487, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /events (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7faed051b610>: Failed to establish a new connection: [Errno 111] Connection refused',))
Input:
+- - 1 1 1 - - - 1 1 1 - - - - - - - - - - -+
+- - 1 1 1 1 - 1 1 1 - - - - - - - - - - - -+
+- - - 1 - - 1 1 1 1 - - - - - - - - - - - -+
+- 1 - 1 - 1 1 1 - - 1 - - - - - - - - - - -+

应该如何将您的程序作为API使用?

我们希望在学术项目中使用您的代码，需要的功能是基于原有状态运行一步DNC，返回状态和输出。请问哪一个文件实现了这样的功能？

Simplest recommended way to add new envs

Hi Jingweiz,

this a great implementation of the DNC, thanks a lot for sharing it. I was wondering what's the recommended/simplest way to add new environments, (from OpenAI)? I guess it would just mean adding them to ./utils/factory.py?

There's an interesting paper I've been reading, Bridging the Gap Between Value and Policy Based Reinforcement Learning, which compares some A3C like algorithms on the six OpenAI algorithmic tasks.

Seems like it would be a nice test for the DNC to train it using A3C on these tasks - I'm interested in both the extra benefit of external memory, and also different ways of training the DNC?

Bidirectional DNC

Hello, thank you for this package it has been very helpful to start experimenting with DNCs. I tried to implement bidirectional version of the circuit by using two separate lstm controllers for forward and backward pass respectively, one thing I am not sure of is whether it makes sense to concatenate read vectors for both directions along with hidden layer activations for the input at time T before feeding them to the linear layer. I would appreciate if you could provide feedback:
https://github.com/Rahim16/pytorch-dnc/blob/master/core/circuits/bi_circuit.py

Thank you!
Rahim

pytorch-dnc-Question

Hi Dear
Thank you for sharing code of NTM-DNC on github & thanks for the your explanations, I have a problem when trying to test the code, can you help me?
the error is: NameError: name 'raw_input' is not defined
how can i fixed this?

your kind attention is highly appreciated in advance.
best regards

Have you tried the shortest path experiment?

Very nice code. May I ask if you have tried the shortest path experiment described in the paper?

Thanks.