abaheti95/deep-learning

I'll put all the valuable tutorials and starter codes of deep learning here.

deep-learning's Issues

Keras on GPU

Hi,

Thanks for sharing your code, I was looking for an implementation of Word2Vec purely in Keras!

I was wondering if the Keras one was running on GPU or no?

Thanks :)

Thomas

Why G.vocab_size+3 but not G.vocab_size

Hi, as far as I can see the embedding layer should be the vocab_size as input. But in cbow_model.py of Keras

shared_embedding_layer = Embedding(input_dim=(G.vocab_size+3), output_dim=G.embedding_dimension, weights=[embedding])

Where is the 3 from?

Does not compile without errors in Python 3.6.4 64 bits

Hi,
I tried to compile the Word2Vec CBOW example at e.g.
https://github.com/abaheti95/Deep-Learning/blob/master/word2vec/keras/cbow_model.py
but that failed.
Could you please check.
Python 3.6.4 in 64 bits.
Thanks

E.g.

Using TensorFlow backend.
Traceback (most recent call last):
File "ddd.py", line 5, in
from keras.utils.np_utils import accuracy
ImportError: cannot import name 'accuracy'

I tried to
pip3.6.exe install np_utils
pip3.6.exe install future
pip3.6.exe install tensorflow
pip3.6.exe install numpy

===

Further also e.g. the line

import global_settings as G

gives a can not be found error.

Shapes do not match

Hi, me again sorry!

I get this error when I run your code:
Traceback (most recent call last):

File "cbow_model.py", line 49, in <module>
  word_context_product = merge([word_embedding, cbow], mode='dot')
File "/usr/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1680, in merge
  name=name)
File "/usr/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1301, in __init__
  self.add_inbound_node(layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
  Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/site-packages/keras/engine/topology.py", line 172, in create_node
  output_tensors = to_list(outbound_layer.call(input_tensors, mask=input_masks))
File "/usr/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1409, in call
  output = K.batch_dot(l1, l2, self.dot_axes)
File "/usr/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 908, in batch_dot
  out = tf.matmul(x, y, adjoint_a=adj_x, adjoint_b=adj_y)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 1855, in matmul
  a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1454, in _mat_mul
  transpose_b=transpose_b, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
  op_def=op_def)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2397, in create_op
  set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1757, in set_shapes_for_outputs
  shapes = shape_func(op)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1707, in call_with_requiring
  return call_cpp_shape_fn(op, require_shape_fn=True)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
  debug_python_shape_fn, require_shape_fn)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
  raise ValueError(err.message)
ValueError: Shape must be rank 2 but is rank 3 for 'MatMul' (op: 'MatMul') with input shapes: [?,1,100], [?,100].

Embeddings layers indeed return 3D tensors of shape (samples, sequence_length, embedding_dim)...