domerin0/neural-sentiment

A sentiment analyzer using deep rnns, built with TensorFlow

neural-sentiment's Issues

tf.pack() issue while building the model..

Hi Dominik,

Thanks very much for this great tool. I tried to use this tool kit and i am ending up with an error. I have all the dependencies.
The code downloads the database, builds the vocab and 4 data files in processed directory and than when i am trying to build a model it give the following error. Please can you suggest what might be the problem..

Traceback (most recent call last):
File "train.py", line 149, in
main()
File "train.py", line 69, in main
model = createModel(sess, vocab_size)
File "train.py", line 123, in createModel
FLAGS.learning_rate, FLAGS.lr_decay_factor)
File "/home/neural-sentiment/models/sentiment.py", line 59, in init
concat_states = tf.pack(encoder_state)
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 228, in pack
return gen_array_ops._pack(values, name=name)
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 523, in _pack
return _op_def_lib.apply_op("Pack", values=values, name=name)
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 357, in apply_op
(input_name, op_type_name, values))
TypeError: Expected list for 'values' argument to 'Pack' Op, not Tensor("RNN/concat_199:0", shape=(?, 200), dtype=float32).

regards
Kannadaraj

Problem with tensorflow version 0.12 head

Hi, I have installed tensorflow from source.
In the sentiment.py I had to change tf.contrib.rnn.LSTMCell, tf.contrib.rnn.DropoutWrapper, tf.contrib.rnn.MultiRNNCell at line 75. But cannot find any replacement for rnn.rnn. I have tried tf.nn.rnn but it gave an error "'module' object has no attribute 'rnn' ". I have also tried tf.contrib.rnn but got an error "'module' object is not callable "

memory is blowing up

This is only occurring on the dev branch, but it seems to freeze and take up all the available memory. I probably have some stupid mistake somewhere, and will fix it shortly.

Training is not working?

I've just cloned your repo, downloaded the nltk tokenizer dataset and just tried to train the model using the default config. And that's what I've got after training it for 3 hours on CPU

The accuracy is just 50% all the time. Am I doing something wrong or maybe something has changed?
Output of the python --version is
Python 2.7.13 :: Anaconda custom (64-bit)
The TensorFlow version is 1.2.1
Ubuntu 16.04

Missing util/hyperparams?

Hi Domnik,
I am wondering if the hyper parameters should be added from our side or is it missing from the repository (under util/hyperparams.py). Thank you for the code.

Steven

padding all sequences to same length is inefficient

Will add bucketing to resolve.

Tensorflow version

Which tensorflow version should we use with this code?

Code does not run with the latest Tensorflow version.

Mostly it breaks at wherever there is RNN cell. I am trying to fix it, but too many changes. I would suggest you to keep the repo updated if possible.

ValueError: setting an array element with a sequence

I was trying to use this model. When I tried on my local system, it worked correctly. Same thing I tried to run on aws server, it gave me

ValueError: setting an array element with a sequence

Error code:

WARNING:tensorflow:<tensorflow.python.ops.rnn_cell.LSTMCell object at 0x7f56e6c2cb10>: The input_size parameter is deprecated.
Traceback (most recent call last):
  File "train.py", line 194, in <module>
    main()
  File "train.py", line 63, in main
    model = create_model(sess, hyper_params, vocab_size)
  File "train.py", line 124, in create_model
    hyper_params["batch_size"])
  File "/home/datametica/karim/deeplearning/neural-sentiment/models/sentiment.py", line 73, in __init__
    self.initial_state = tf.zeros([self.batch_size, self.cell.state_size])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1184, in zeros
    shape = ops.convert_to_tensor(shape, dtype=dtypes.int32, name="shape")
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 657, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 180, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 163, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 354, in make_tensor_proto
    nparray = np.array(values, dtype=np_dt)
ValueError: setting an array element with a sequence.

I thought it is due to numpy version. so I tried changing it, but no help.

What I found is, this code works well with tensorflow 0.8.0 .

If you install latest tensorflow and try this code, it will give this error.

So I uninstalled latest version and installed 0.8.0, now again it works fine.

checkpoint names

I made some changes to your code to run it on Tensorflow 1.0.1. Training works fine, and I see checkpoints getting saved in the data/checkpoints/ folder as follows:

sentiment0.501447366256.ckpt-50.data-00000-of-00001
sentiment0.501447366256.ckpt-50.index
sentiment0.501447366256.ckpt-50.meta
sentiment0.494999998494.ckpt-100.data-00000-of-00001
sentiment0.494999998494.ckpt-100.index
sentiment0.494999998494.ckpt-100.meta

However, in the checkpoints file, the checkpoint names are saved as follows:

model_checkpoint_path: "sentiment0.494999998494.ckpt-100"
all_model_checkpoint_paths: "sentiment0.501447366256.ckpt-50"
all_model_checkpoint_paths: "sentiment0.494999998494.ckpt-100"

Now when I run python sample.py , it says model not found, please check that your checkpoint_dir is right. Why is this happening? I tried adding ".meta" or ".index" to the end of model_checkpoint_path. Then the model is found but there are other errors and the model doesn't get loaded. Any idea why this is happening?

There is no restoreHyperParameters function in the sample.py

There is no restoreHyperParameters function in the sample.py
def loadModel(session, vocab_size):
hParams = restoreHyperParameters()
model = models.sentiment.SentimentModel(int(hParams[0]), int(hParams[1]),
float(hParams[2]), int(hParams[3]), int(hParams[4]), int(hParams[5]),
float(hParams[6]),float(hParams[7]) ,True)
ckpt = tf.train.get_checkpoint_state(FLAGS.checkpoint_dir)
if ckpt and gfile.Exists(ckpt.model_checkpoint_path):
print ("Reading model parameters from {0}".format(ckpt.model_checkpoint_path))
model.saver.restore(session, ckpt.model_checkpoint_path)
else:
print ("Double check you got the checkpoint_dir right...")
print ("Model not found...")
model = None
return model

logistics regression layer not properly implemented

I will need to fix the logistic regression layer to properly output the label.

Dropout used incorrectly!

during the test the dropout_keep_prob parameter is not set to 1 as it should, the dropout_keep_prob parameter shouldn't be constant but a placeholder or at least a tf.Variable

Model not found...

Hi @inikdom
I've test the model with this command(python sample.py --text="she is ugly.")
but I got this error
/Projects/neural-sentiment$ python sample.py --text="very good"
2018-05-18 23:51:37.258393: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-18 23:51:37.258419: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-18 23:51:37.258434: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-05-18 23:51:37.258441: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-18 23:51:37.258456: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Double check you got the checkpoint_dir right...
Model not found...

How can I fix it!