visionlearninggroup/mule

Implementation of "MULE: Multimodal Universal Language Embedding"

mule's Issues

Errors in Runing the code

Hi @donhk1011
Thank you for the great work and this code. I am trying to run the code on Google colab with tensorflow 1.13.1 and python 3 but the code is giving me following errors:

Traceback (most recent call last):
  File "/content/drive/MyDrive/Research/MULE-master/train.py", line 214, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/content/drive/MyDrive/Research/MULE-master/train.py", line 168, in main
    vecs, restore_path = process_epoch(epoch, train_data_loader, restore_path, vecs, pretrain_univeral_embedding)
  File "/content/drive/MyDrive/Research/MULE-master/train.py", line 44, in process_epoch
    train_step = optim.minimize(loss, global_step=global_step)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/optimizer.py", line 403, in minimize
    grad_loss=grad_loss)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/optimizer.py", line 512, in compute_gradients
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 664, in gradients
    unconnected_gradients)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 965, in _GradientsHelper
    lambda: grad_fn(op, *out_grads))
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 420, in _MaybeCompile
    return grad_fn()  # Exit early
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 965, in <lambda>
    lambda: grad_fn(op, *out_grads))
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_grad.py", line 487, in _GatherNdGrad
    ref_grad = array_ops.scatter_nd(indices, grad, ref_shape)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 7825, in scatter_nd
    "ScatterNd", indices=indices, updates=updates, shape=shape, name=name)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Indices and updates specified for empty output shape for 'gradients/GatherNd_grad/ScatterNd' (op: 'ScatterNd') with input shapes: [900,2], [900,1024], [3] and with input tensors computed as partial shapes: input[2] = [900,0,1024].

I think that would probably because of tensorflow version and python version. Therefore do you have code compatible with python 3 and some above version of tensorflow? Could you please help me in this regard? Thank you.