bstriner/bayesian_dense

Bayesian Weight Uncertainty Dense Layer for Keras

bayesian_dense

Bayesian Weight Uncertainty for Keras

The BayesianDense layer is a Dense layer parameterized by a weight distribution, instead of a point estimate. Each BayesianDense layer learns a Gaussian distribution over weights and biases that can be regularized.

VariationalRegularizer is an exemplary regularizer calculating -0.5 * mean(1 + p - K.exp(p)) where p is log of sigma squared. This is just a simple regularizer. Please experiment and let me know any interesting variations.

My implementation of the following paper:

Blundell et. al., Weight Uncertainty in Neural Networks, https://arxiv.org/pdf/1505.05424.pdf

bayesian_dense's Issues

Many errors when using Keras 2.2 version

want to replace below Dense layer with BayesianDense.

ct_input = Input(shape=(PATCH_HEIGHT, PATCH_WIDTH, 1))
pet_input = Input(shape=(PATCH_HEIGHT, PATCH_WIDTH, 1))

x = concatenate([ct_model, pet_model])
x = Dense(864, activation='relu')(x)
x = Dense(288, activation='relu')(x)
output = Dense(1, activation='sigmoid')(x)

I got many errors. E,g: ImportError: cannot import name... Init and Build
Do you have any example to use with relevant output graphs.

Keras 2.0

Have you thought about updating your code for keras 2?

Unable to execute example

TypeError Traceback (most recent call last)
in ()
73 decay = 0.96
74 lr = 1e-3
---> 75 m=model()
76 m.summary()
77 log = []

in model(hidden_dim, input_dim, sigma_regularization, mu_regularization, k, activation)
44 h2 = layer(hidden_dim, activation)
45 y = layer(k, 'softmax')
---> 46 _y = y(h2(h1(_x)))
47
48 m = Model(_x, _y)

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in call(self, inputs, **kwargs)
429 'You can build it manually via: '
430 'layer.build(batch_input_shape)')
--> 431 self.build(unpack_singleton(input_shapes))
432 self.built = True
433

in build(self, input_shape)
76
77 self.W_mu = self.init((input_dim, self.output_dim),
---> 78 name='{}_W_mu'.format(self.name))
79 self.W_log_sigma = self.init_sigma((input_dim, self.output_dim),
80 name='{}_W_log_sigma'.format(self.name))

TypeError: call() got an unexpected keyword argument 'name'