Comments (3)
For the initializer you would need to add arguments for your feature sizes, and create a new Embedding layer for each discrete feature. In the forward() method you would add arguments for the extra features, forward them through the relevant embedding layers, and concatenate them all into one vector for the RNN. The RNN also would have an input size of 3 * hidden_size
because of the three embeddings added together (or you could create different embedding sizes for each feature).
Overall it should look something like this:
class EncoderRNN(nn.Module):
def __init__(..., word_size, feature2_size, feature3_size, hidden_size, ...):
...
self.word_embedding = nn.Embedding(word_size, hidden_size)
self.feature2_embedding = nn.Embedding(feature2_size, hidden_size)
self.feature3_embedding = nn.Embedding(feature3_size, hidden_size)
# Note: * 3 because the above 3 embeddings will be concatenated
self.gru = nn.GRU(hidden_size * 3, hidden_size, n_layers, dropout=self.dropout, bidirectional=True)
def forward(self, word_seqs, feature2_seqs, feature3_seqs, input_lengths, hidden=None):
# Note: we run this all at once (over multiple batches of multiple sequences)
word_embedded = self.word_embedding(word_seqs)
feature2_embedded = self.feature2_embedding(feature2_seqs)
feature3_embedded = self.feature3_embedding(feature3_seqs)
combined = torch.cat((word_embedded, feature2_embedded, feature3_embedded), 2)
packed = torch.nn.utils.rnn.pack_padded_sequence(combined, input_lengths)
...
from practical-pytorch.
The simplest way is to concatenate features into a single input vector. However this only works if your RNN takes vector input, not discrete inputs (LongTensor) through an embedding layer. In that case you would want to concatenate your extra features after the input is embedded. This can most easily be achieved by using an extra input argument in your encoder and using torch.cat
(see the category input in https://github.com/spro/practical-pytorch/blob/master/conditional-char-rnn/conditional-char-rnn.ipynb for a simple example, though that does not use an embedding layer).
If your features are also discrete you would want multiple embedding layers, one for each, and concatenate all the results.
The quoted section is more about input and target sequence length, not feature size.
from practical-pytorch.
@spro Thanks much for your inputs. I have written a blog post on the same -
https://iamsiva11.github.io/extra-features-seq2seq/. Hope it'll be useful for many others.
from practical-pytorch.
Related Issues (20)
- Issue on Windows
- Seq-seq not working for creating chatbot
- How to save and load train model and use it for evaluation HOT 2
- Link for Series 2 - RNNs for time-series data
- Question about Luong Attention Implementation HOT 7
- can't import torch HOT 2
- The link for Teacher Forcing in "Translation with a Sequence to Sequence Network and Attention" is broken
- Error in BahdanauAttnDecoderRNN HOT 1
- Issues in your tutorial on Classifying Names with a Character-Level RNN
- I can't calculate the score of attention in Seq2Seq Translation. HOT 2
- Error in practical-pytorch/seq2seq-translation/seq2seq-translation-batched.ipynb
- Question from character level RNN classifier, why not use the hidden state across epochs? HOT 1
- RuntimeError: 1D tensors expected, got 2D, 2D tensors HOT 1
- May I know how to support a new sentence translation?
- seq2seq: Replace the embeddings with pre-trained word embeddings such as word2vec
- About seq2seq-translation-batched.py RuntimeError HOT 1
- Wrong implementation of attention mechanism in pytorch tutorials
- FileNotFoundError: [Errno 2] No such file or directory: 'char-rnn-classification.pt'
- small format issue
- Implicit dimension choice for log_softmax has been deprecated while running 'python train.py'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from practical-pytorch.