AdaGradOptimizer on GPU

[tensorflow]

AdaGradOptimizer on GPU

[acelove]

I have the same problem now and I don't know why. So I can only modify the definition of the Embedding class in file tensorflow_fold/blocks/layers.py. Because there is a line
return tf.gather(self._weights, batch)
So I force the variable "self._weights" to use cpu by
with tf.device("/cpu:0"): self._weights = tf.get_variable('weights',self._weights_shape, initializer=self._initializer, trainable=self._trainable)
in function _create_variables.
It works.
By the way, you can find the path of the file layers.py by command
locate tensorflow_fold/blocks/layers.py

[delesley]

This sounds like an error somewhere in the TensorFlow stack, not the TensorFlow Fold stack, so I have no idea what's wrong. You might try seeing if someone on the main TensorFlow site has any ideas. There were some bugs a while back with the combination of tf.gather and Adagrad, but they should have been fixed by now. If you are feeling adventurous, you could try to see whether you can reproduce the problem using Adagrad and tf.gather without the Fold library. Sorry I couldn't be of more help...

…

On Wed, Feb 15, 2017 at 2:24 AM, arwhirang ***@***.***> wrote: Hello, When I executed the same code posted on the The sentiment notebook <https://github.com/tensorflow/fold/blob/master/tensorflow_fold/g3doc/sentiment.ipynb>, following error message is displayed. InvalidArgumentError: Cannot assign a device to node 'Adagrad/update_word_embedding/weights/Unique': Could not satisfy explicit device specification '' because the node was colocated with a group of nodes that required incompatible device '/job:localhost/replica:0/task:0/GPU:0' Colocation Debug Info: Colocation group had the following types and devices: SparseApplyAdagrad: CPU UnsortedSegmentSum: GPU CPU StridedSlice: GPU CPU Shape: GPU CPU Unique: GPU CPU VariableV2: GPU CPU Const: GPU CPU [[Node: Adagrad/update_word_embedding/weights/Unique = Unique[T=DT_INT32, ***@***.***_embedding/weights"], out_idx=DT_INT32](gradients/ while/Function_19/word_embedding_1/Gather/Enter_grad/b_acc_6)]] I found that when I change the adagrad to GradientDescentOptimizer or AdamOptimizer, the code works, but the accuracy of the model is inferior than the adagrad version. I found that Adagrad is not working on gpu ... <tensorflow/tensorflow#1570> any help, please? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#10>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGGbTRuVxeBI7GoXxMFKKn4IrkNR3T4gks5rctJrgaJpZM4MBiax> .

-- DeLesley Hutchins | Software Engineer | [email protected] | 505-206-0315

[arwhirang]

Thanks for reply.
I have tested the code from the following url <blog_tensorflow_variable_sequence_classification.py>https://gist.github.com/danijar/3f3b547ff68effb03e20c470af22c696 (of course, with minor changes regarding the tensorflow ver1.0). The code use Tensorflow only. And the code contains the tf.gather operation and I confirm that the Adagrad optimizer works fine there.

Actually, I am at a loss. Could you tell me how do I ask a question to explain this problem in detail, assuming that I ask this question to Tensorflow community?

[delesley]

Hmm... that is a mystery, then. BTW, the question you asked me was clear enough. :-) ​

[chuneli]

yes, i have the same problem now...