jonvet/thesis

MSc project: Inferring Sentence Features from Sentence Embeddings

thesis's Issues

Suggestions: may be a error exists in model define codes.

Hi, firstly, thanks for your beautiful codes to display InferSent model in tensorflow.
There exists a problem which bothers me.
You use the same cell for forward and backward cell in BiLstm, as follows:

59             cell = tf.contrib.rnn.LSTMCell(self.para.hidden_size)
60             cell = tf.contrib.rnn.DropoutWrapper(cell, input_keep_prob =
61                                                  self.para.keep_prob_dropout)

But I hold the thought that forward cell and backward should be different so that they can learn sentences features independently, as follows:

 63             cell_fw = tf.contrib.rnn.LSTMCell(self.para.hidden_size)
 64             cell_fw = tf.contrib.rnn.DropoutWrapper(cell_fw, input_keep_prob =
 65                                                  self.para.keep_prob_dropout)
 66             cell_bw = tf.contrib.rnn.LSTMCell(self.para.hidden_size)
 67             cell_bw = tf.contrib.rnn.DropoutWrapper(cell_bw, input_keep_prob =
 68                                                  self.para.keep_prob_dropout)
 69             s1_sentences_states, s1_last_state = tf.nn.bidirectional_dynamic_rnn(
 70                 cell_fw, cell_bw,
 71                 inputs = self.s1_embedded,
 72                 sequence_length = self.s1_lengths,
 73                 dtype = tf.float32,
 74                 scope = varscope)

After modified, with 10 epochs, the performance seems little better than before.
Do you agree with me? If not, please tell me why, thanks ^_^