mirceamironenco/bayesianrecurrentnn

Implementation of Bayesian Recurrent Neural Networks by Fortunato et. al

bayesianrecurrentnn's Issues

AttributeError when running code

Hi, I am trying to run this code with tensorflow 1.3.0 and Python3.6

I get a similar error to a separate issue raised:

Traceback (most recent call last):
  File "bayesian_rnn.py", line 560, in <module>
    run()
  File "bayesian_rnn.py", line 510, in run
    m = PTBModel(is_training=True, config=config, input_=train_input)
  File "bayesian_rnn.py", line 229, in __init__
    bbb_bias=config.bbb_bias))
  File "bayesian_rnn.py", line 140, in __init__
    **self.name = name**
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1639, in __setattr__
    super(Layer, self).__setattr__(name, value)
AttributeError: can't set attribute

The problematic line self.name = name comes from the BayesianLSTM class. This is the same error as the other issue however I am using Tensorflow 1.3.0 so the issue appears to be around using Python 3.6 instead of Python3.5. Is there a way to amend to code so that it is compatible with Python 3.6.

question of define prior

HI, nice work!
I was wondering in bayesian_rnn.py Line 112
why define the prior as pi*math.exp(sigmaone)+(1-pi)math.exp(sigmatwo)
I think reference to your paper, maybe
gaussian1 = torch.distributions.Normal(0,sigma1)
gaussian2 = torch.distributions.Normal(0,sigma2)
prior = pigaussian1+(1-pi)*gaussian2
is correct.
So why this idea?

About the KL loss

Good job! Thanks for sharing it!
I have some issues when I use BayesianRNN of your project into my own work. I found that the KL loss is so large which is up to 9.00*10^8 and my prediction loss is 20.00 which is normal, for each time step. So the training process can not continue because the KL loss is not regular. So I dont know the reson why this issue occurs.
I would appreciate it if you can give me some kinds of tips for fixing it or any suggestions.

error with "name" in class BayesianLSTM

I tried to reproduce the results of given code and got the error like this:

can you help figure out why this happened? I haven't started to change the code yet so this's the original one.