saketd403/visualizing-and-understanding-convolutional-neural-networks

Visualizing-and-Understanding-Convolutional-neural-networks

This is a straight from scratch implementation of paper "Visualizing and Understanding Convolutional neural networks"-Matthew D. Zeiler and Rob Fergus (https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf)

This implementation is using keras with tensorflow backend.

The implementation is mainly divided into two parts-

1. Feature visualization with Deconvnet.
2. Occlusion Sensitivity.

Part 1- Feature visualization with Deconvnet This is described in notebook "Feature visualization using Deconvnets in Keras". Here the term deconvolution refers to transpose convolution.

Original image -

Features detected in feature map 127 of convolutional layer 3 of block 3 of VGG16 model-

Part 2- Occlusion sensitivity This experiment is carried out to test if the convnet is actualy locating and detecting the desired object in image and not some pattern or object in background.

The experiment is described in notebook- "Occlusion experiment".The experiment is carried out on MNIST hanwritten digit data set.

Original image -

Occlusion map -

visualizing-and-understanding-convolutional-neural-networks's Issues

deconv = visualize(model, img_array, layer_name, feature_to_visualize, visualize_mode)

when exec
deconv = visualize(model, img_array, layer_name, feature_to_visualize, visualize_mode)
There is an InvalidArgumentError
`InvalidArgumentError Traceback (most recent call last)
in
1 deconv = visualize(model, img_array,
----> 2 layer_name, feature_to_visualize, visualize_mode)

in visualize(model, data, layer_name, feature_to_visualize, visualize_mode)
29 deconv_layers[0].up(data)
30 for i in range(1, len(deconv_layers)):
---> 31 deconv_layers[i].up(deconv_layers[i - 1].up_data)
32
33 output = deconv_layers[-1].up_data

in up(self, data, learning_phase)
34 def up(self, data, learning_phase = 0):
35 #Forward pass
---> 36 self.up_data = self.up_func([data, learning_phase])
37 self.up_data=np.squeeze(self.up_data,axis=0)
38 self.up_data=numpy.expand_dims(self.up_data,axis=0)

~/anaconda3/envs/tfEnv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in call(self, inputs)
2713 return self._legacy_call(inputs)
2714
-> 2715 return self._call(inputs)
2716 else:
2717 if py_any(is_tensor(x) for x in inputs):

~/anaconda3/envs/tfEnv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
2673 fetched = self._callable_fn(*array_vals, run_metadata=self.run_metadata)
2674 else:
-> 2675 fetched = self._callable_fn(*array_vals)
2676 return fetched[:len(self.outputs)]
2677

~/anaconda3/envs/tfEnv/lib/python3.6/site-packages/tensorflow/python/client/session.py in call(self, *args, **kwargs)
1380 ret = tf_session.TF_SessionRunCallable(
1381 self._session._session, self._handle, args, status,
-> 1382 run_metadata_ptr)
1383 if run_metadata:
1384 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/anaconda3/envs/tfEnv/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in exit(self, type_arg, value_arg, traceback_arg)
517 None, None,
518 compat.as_text(c_api.TF_Message(self.status.status)),
--> 519 c_api.TF_GetCode(self.status.status))
520 # Delete the underlying status object from memory otherwise it stays alive
521 # as there is a reference to status from this from the traceback due to

InvalidArgumentError: transpose expects a vector of size 6. But input(1) is a vector of size 4
[[Node: conv2d_84/Conv2D-0-TransposeNHWCToNCHW-LayoutOptimizer = Transpose[T=DT_FLOAT, Tperm=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](_arg_input_85_0_0/_1345, PermConstNHWCToNCHW-LayoutOptimizer)]]`

Change the units parameter of Dense in self.down_func to integer instead of tuple

Hi @saketd403 ,

Thank you for your wonderful work :)) However, there is a minor error in the class DDense (Feature+visualization+using+Deconvnets+in+Keras.ipynb) that I think you should fix.
Specifically,

output = keras.layers.Dense(self.input_shape[1:], 
kernel_initializer=tf.constant_initializer(W), bias_initializer=tf.constant_initializer(b))(input)

should be changed to

output = keras.layers.Dense(self.input_shape[1],
kernel_initializer=tf.constant_initializer(W), bias_initializer=tf.constant_initializer(b))(input)

in the definition of self.down_func.

Vinh

No file 'C:\Users\USER\Desktop\inverting convolutional layers\Max pooling\model.pkl'

When I ran the file 'Occlusion+experiment', I met the problem in the following pictures. Could you please upload the file 'model.pkl'?

Give credit to owner's repo

Hi @saketd403 ,

Thank you so much for your wonderful work. I tried days to find an actual implementation of DeconvNN, and yours solved my problem.

However, during my searching, I found that yours may be adapted from the repo of jalused. Though his work is not compatible with the current Keras version, the coding style is quite similar.

I suggest you add a comment to give him a credit if that's the case :)

Vinh