kamo-naoyuki/pytorch_convolutional_rnn

PyTorch implementation of Convolutional Recurrent Neural Network

pytorch_convolutional_rnn

The pytorch implemenation for convolutional rnn is alreaedy exisitng other than my module, for example.

• https://github.com/ndrplz/ConvLSTM_pytorch
• https://github.com/jacobkimmel/pytorch_convgru

However, there are no modules supporting neither variable length tensor nor bidirectional rnn.

I implemented AutogradConvRNN by referring to AutogradRNN at https://github.com/pytorch/pytorch/blob/master/torch/nn/_functions/rnn.py, so my convolutional RNN modules have similar structure to torch.nn.RNN and supports the above features as it has.

The benefit of using AutogradConvRNN is not only that it enables my modules to have the same interface as torch.nn.RNN, but makes it very easy to implement many kinds of CRNN, such as CLSTM, CGRU.

Require

• python3 (Not supporting python2 because I prefer type annotation)
• pytorch0.4.0, python1.0.0

Feature

• Implemented at python level, without any additional CUDA kernel, c++ codes.
• Convolutional RNN, Convolutional LSTM, Convolutional Peephole LSTM, Convolutional GRU
• Unidirectional, Bidirectional
• 1d, 2d, 3d
• Supporting PackedSequence (Supporting variable length tensor)
• Supporting nlayers RNN and RNN Cell, both.
• Not supporting different hidden sizes for each layers (But, it is very easy to implement it by stacking 1-layer-CRNNs)

Example

• With pack_padded_sequence

import torch
import convolutional_rnn
from torch.nn.utils.rnn import pack_padded_sequence

in_channels = 2
net = convolutional_rnn.Conv3dGRU(in_channels=in_channels,  # Corresponds to input size
                                  out_channels=5,  # Corresponds to hidden size
                                  kernel_size=(3, 4, 6),  # Int or List[int]
                                  num_layers=2,
                                  bidirectional=True,
                                  dilation=2, stride=2, dropout=0.5)
length = 3
batchsize = 2
lengths = [3, 1]
shape = (10, 14, 18)
x = pack_padded_sequence(torch.randn(length, batchsize, in_channels, *shape), lengths, batch_first=False)
h = None
y, h = net(x, h)

• Without pack_padded_sequence

import torch
import convolutional_rnn
from torch.nn.utils.rnn import pack_padded_sequence

in_channels = 2
net = convolutional_rnn.Conv2dLSTM(in_channels=in_channels,  # Corresponds to input size
                                   out_channels=5,  # Corresponds to hidden size
                                   kernel_size=3,  # Int or List[int]
                                   num_layers=2,
                                   bidirectional=True,
                                   dilation=2, stride=2, dropout=0.5,
                                   batch_first=True)
length = 3
batchsize = 2
shape = (10, 14)
x = torch.randn(batchsize, length, in_channels, *shape)
h = None
y, h = net(x, h)

• With Cell

import torch
import convolutional_rnn
cell = convolutional_rnn.Conv2dLSTMCell(in_channels=3, out_channels=5, kernel_size=3).cuda()
time = 6
input = torch.randn(time, 16, 3, 10, 10).cuda()
output = []
for i in range(time):
    if i == 0:
        hx, cx = cell(input[i])
    else:
        hx, cx = cell(input[i], (hx, cx))
    output.append(hx)

pytorch_convolutional_rnn's Issues

Help for using the code in practice

Hi man, great job!
I'm interested in using your code in my project, but although I'm used to the theory behind RNN's and specially GRU's, I'm not skilled enough for doing the things work in practice. Do you know any other project that has used your code as baseline, performing steps like training and inference? If so, it would be very helpful. Thanks you so much!

Input data's shape for sending to network

Hi,

First I want to thank you for sharing such a great work. I am having some issue using the net, I hope you can manage sometime to explain it for me.

My input shape is like: (batch_size, number_of_features, image_rows, image_cols, image_sequence_length) - frame volume type.
I want to use the convolutional lstm layer for this. Which layer should I use and how can I translate my input into shape that the layer accept.

Thank you.

fail to set batch_first == True

Thanks for sharing your code. When I use this code, I find that when I set the batch_first == True, the net still with batch_first == False version. Any ideas?

Thanks again

stride was not used