dennybritz/tf-rnn

Practical Examples for RNNs in Tensorflow

tf-rnn's Issues

How to mix inference with training data based on queues

Hi, thanks for the tutorial.

This is more of a question so sorry if it's not in the right place.

I'm training a language model using the queue and TFRecord method that you've described in the tutorials. I am able to feed in the batched / padded data into the TensorFlow graph and train the model.

Something like:

...
# queue for the data
batched_data = tf.train.batch(
    tensors=[y],
    batch_size=5,
    dynamic_pad=True,
    name="y_batch"
)
...
# specify the model
embedding = tf.Variable(shape=(vocab_size, hidden_units), dtype=tf.float32)
embedded_input = tf.nn.embedding_lookup(
         embedding, batched_data, name="embedded_input")
...
# run the model
sess.run([train_op], feed_dict={})
...

I'd like to check the language model performance by generating words from the trained model. I would run the model however many steps to complete the sentence.

This requires me to feed in new data that's not part of the training data queue.

Previously, it was easy to use feed_dict and feed the new data in. But my model graph is entirely dependent on the batched_data above and so I wasn't sure how to add arbitrary data separate from or on top of the queue. This would be important at inference time as well.

Thanks!

How to get the last output?

In dynamic_rnn.ipynb, you can get the last ouput by result[0]["outputs"][1,6]
but in most cases, the last output index could be variable, How to get the last outputs in a batch?

one-padding in dynamic_rnn.ipynb

Hi,
Thanks for your excelent tutorial! I learned a lot from your code and the post.

One question is

tf.reset_default_graph()

# Create input data
X = np.random.randn(2, 10, 8)

# The second example is of length 6 
X[1,6,:] = 0
X_lengths = [10, 6]

The second example is of length 6. So I think if there is X[1, 6:] = 0 rather than X[1, 6, :] = 0?