v0lta/complex-gated-recurrent-neural-networks

Complex domain recurrent neural network gating and Stiefel-manifold optimization in TensorFlow, Neural Information Processing Systems (NeurIPS) 2018.

complex-gated-recurrent-neural-networks's Issues

Does StiefelGatedRecurrentUnit implements return_sequence?

In the self-defined RNN subclass-StiefelGatedRecurrentUnit, does it implements the function of parameter-return_sequence as in tf.keras.layers.LSTM?

Not an Issue

Hi Moritz, I was just curious to know if you have a PyTorch implementation of this repo.

TypeError: complex_matmul() got an unexpected keyword argument 'bias_init_c'

I have git cloned the repo and run python bonn_eval_gate_diff.py, but following error produced:

Traceback (most recent call last):
  File "bonn_eval_gate_diff.py", line 83, in <module>
    single_gate)
  File "/home/tang/Code/A20/Complex-gated-recurrent-neural-networks/synthetic_experiments.py", line 157, in main
    y_hat = tf.nn.dynamic_rnn(cell, x, dtype=tf.float32)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/rnn.py", line 707, in dynamic_rnn
    dtype=dtype)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/rnn.py", line 916, in _dynamic_rnn_loop
    swap_memory=swap_memory)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3501, in while_loop
    return_same_structure)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3012, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2937, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3456, in <lambda>
    body = lambda i, lv: (i + 1, orig_body(*lv))
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/rnn.py", line 884, in _time_step
    (output, new_state) = call_cell()
  File "/home/tang/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/rnn.py", line 870, in <lambda>
    call_cell = lambda: cell(input_t, state)
  File "/home/tang/Code/A20/Complex-gated-recurrent-neural-networks/custom_cells.py", line 930, in __call__
    bias_init=bias_init)
  File "/home/tang/Code/A20/Complex-gated-recurrent-neural-networks/custom_cells.py", line 859, in double_memory_gate
    bias_init_r=bias_init)
TypeError: complex_matmul() got an unexpected keyword argument 'bias_init_c'

Performance issues in custom_optimizer.py

Hello, I found that in custom_optimizer.py, init_rms and init_eps are repeatedly created in the loop. As they ratain same in each iteration, I think they should be moved before the loop to avoid creating redundant nodes in the tf computation graph.

The same issue exists here

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

Numpy version

Can you tell me the version of Numpy you used?