infocusp/tf_cnnvis

CNN visualization tool in TensorFlow

tf_cnnvis's Issues

deepdream_visualization error: invalid reduction dimension

When I call the deepdream_visualization function in the following way for MNIST classification,

random_representation = mnist_data.test.next_batch(1)
ran_x = preprocess_batch(random_representation[0], mean, std)
ran_y = random_representation[1]
feed_dict = {x: ran_x, y: ran_y, is_training: False, keep_probability: 1}
deep_dream = True
if deep_dream:
        layer = 'Conv/convolution'
        start = time.time()
        deepdream_visualization(graph_or_path=tf.get_default_graph(), value_feed_dict=feed_dict, layer=layer,
                                classes=[1, 2, 3, 4, 5, 6, 7, 8, 9],
                                input_tensor=None,
                                path_logdir="C:/Users/bucpau/PycharmProjects/Academy/Logs/",
                                path_outdir="C:/Users/bucpau/PycharmProjects/Academy/Visualization/")
        start = time.time() - start
        print("Total time for deconvolution visualization: {} Success: {}".format(start, is_success))

I get an error:
Traceback (most recent call last): File "C:/Users/bucpau/PycharmProjects/Academy/CNN_MNIST.py", line 156, in <module> main() File "C:/Users/bucpau/PycharmProjects/Academy/CNN_MNIST.py", line 153, in main visualize_layers(test_dict) File "C:/Users/bucpau/PycharmProjects/Academy/CNN_MNIST.py", line 85, in visualize_layers path_outdir="C:/Users/bucpau/PycharmProjects/Academy/Visualization/") File "C:\ProgramData\Anaconda3\lib\site-packages\tf_cnnvis-1.0.0-py3.6.egg\tf_cnnvis\tf_cnnvis.py", line 393, in deepdream_visualization File "C:\ProgramData\Anaconda3\lib\site-packages\tf_cnnvis-1.0.0-py3.6.egg\tf_cnnvis\tf_cnnvis.py", line 138, in _get_visualization File "C:\ProgramData\Anaconda3\lib\site-packages\tf_cnnvis-1.0.0-py3.6.egg\tf_cnnvis\tf_cnnvis.py", line 264, in _visualization_by_layer_name File "C:\ProgramData\Anaconda3\lib\site-packages\tf_cnnvis-1.0.0-py3.6.egg\tf_cnnvis\tf_cnnvis.py", line 317, in _deepdream File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1382, in reduce_mean name=name) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 1364, in _mean keep_dims=keep_dims, name=name) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 767, in apply_op op_def=op_def) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2632, in create_op set_shapes_for_outputs(ret) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1911, in set_shapes_for_outputs shapes = shape_func(op) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1861, in call_with_requiring return call_cpp_shape_fn(op, require_shape_fn=True) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 595, in call_cpp_shape_fn require_shape_fn) File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 659, in _call_cpp_shape_fn_impl raise ValueError(err.message) ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'Mean_1' (op: 'Mean') with input shapes: [?,784], [3] and with computed input tensors: input[1] = <1 2 3>. .

Any ideas on what is happening here? Is it a bug or am I using the function incorrectly? The activation visualization works, but the deconvolution does not with the input feed_dict. The only addition to this function call in comparison to the activation visualization are the parameters classes and layer. Am I setting them correctly?

Weights visualization

Hi, thank you for your work, is of great help!
I'm wondering if your library implements also the view of the weights for each convolutional layer, at different epochs, like Fig.4 and Fig.5 in the paper from Zeiler and Fergus ?

Thank you ^_^

Getting error sudo python setup.py clean

Hi I'm jang

Thank you for sharing your work.

My PC enviornment is Ubuntu 16.04 LTS and I downloads tf_cnnvis on /home/jc using git.

I entered the following command :

pip install tensorflow numpy scipy h5py wget Pillow six scikit-image

git clone https://github.com/InFoCusp/tf_cnnvis.git

sudo pip install setuptools
[sudo] password for jc: The directory '/home/jc/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/jc/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Requirement already satisfied: setuptools in /home/jc/.local/lib/python2.7/site-packages
4.
sudo pip install six
The directory '/home/jc/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/jc/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Requirement

jc@jc-15N540-UT50K:~/tf_cnnvis$ sudo python setup.py install

running install
running bdist_egg
running egg_info
writing requirements to tf_cnnvis.egg-info/requires.txt
writing tf_cnnvis.egg-info/PKG-INFO
writing top-level names to tf_cnnvis.egg-info/top_level.txt
writing dependency_links to tf_cnnvis.egg-info/dependency_links.txt
reading manifest file 'tf_cnnvis.egg-info/SOURCES.txt'
writing manifest file 'tf_cnnvis.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/tf_cnnvis
copying build/lib.linux-x86_64-2.7/tf_cnnvis/utils.py -> build/bdist.linux-x86_64/egg/tf_cnnvis
copying build/lib.linux-x86_64-2.7/tf_cnnvis/__init__.py -> build/bdist.linux-x86_64/egg/tf_cnnvis
copying build/lib.linux-x86_64-2.7/tf_cnnvis/tf_cnnvis.py -> build/bdist.linux-x86_64/egg/tf_cnnvis
byte-compiling build/bdist.linux-x86_64/egg/tf_cnnvis/utils.py to utils.pyc
byte-compiling build/bdist.linux-x86_64/egg/tf_cnnvis/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py to tf_cnnvis.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying tf_cnnvis.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying tf_cnnvis.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying tf_cnnvis.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying tf_cnnvis.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying tf_cnnvis.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/tf_cnnvis-1.0.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing tf_cnnvis-1.0.0-py2.7.egg
Removing /usr/local/lib/python2.7/dist-packages/tf_cnnvis-1.0.0-py2.7.egg
Copying tf_cnnvis-1.0.0-py2.7.egg to /usr/local/lib/python2.7/dist-packages
tf-cnnvis 1.0.0 is already the active version in easy-install.pth

Installed /usr/local/lib/python2.7/dist-packages/tf_cnnvis-1.0.0-py2.7.egg
Processing dependencies for tf-cnnvis==1.0.0
Searching for scikit-image==0.13.1
Best match: scikit-image 0.13.1
Adding scikit-image 0.13.1 to easy-install.pth file
Installing skivi script to /usr/local/bin

Using /home/jc/.local/lib/python2.7/site-packages
Searching for six==1.11.0
Best match: six 1.11.0
Adding six 1.11.0 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for Pillow==5.0.0
Best match: Pillow 5.0.0
Adding Pillow 5.0.0 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for wget==3.2
Best match: wget 3.2
Adding wget 3.2 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for h5py==2.7.1
Best match: h5py 2.7.1
Adding h5py 2.7.1 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for scipy==1.0.0
Best match: scipy 1.0.0
Adding scipy 1.0.0 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for numpy==1.14.1
Best match: numpy 1.14.1
Adding numpy 1.14.1 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for matplotlib==2.1.2
Best match: matplotlib 2.1.2
Adding matplotlib 2.1.2 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for networkx==2.1
Best match: networkx 2.1
Adding networkx 2.1 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for PyWavelets==0.5.2
Best match: PyWavelets 0.5.2
Adding PyWavelets 0.5.2 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for cycler==0.10.0
Best match: cycler 0.10.0
Adding cycler 0.10.0 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for python-dateutil==2.6.1
Best match: python-dateutil 2.6.1
Adding python-dateutil 2.6.1 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for pytz==2018.3
Best match: pytz 2018.3
Adding pytz 2018.3 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for subprocess32==3.2.7
Best match: subprocess32 3.2.7
Adding subprocess32 3.2.7 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for backports.functools-lru-cache==1.5
Best match: backports.functools-lru-cache 1.5
Adding backports.functools-lru-cache 1.5 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for pyparsing==2.2.0
Best match: pyparsing 2.2.0
Adding pyparsing 2.2.0 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Searching for decorator==4.2.1
Best match: decorator 4.2.1
Adding decorator 4.2.1 to easy-install.pth file

Using /home/jc/.local/lib/python2.7/site-packages
Finished processing dependencies for tf-cnnvis==1.0.0
/home/jc/.local/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters

jc@jc-15N540-UT50K:~/tf_cnnvis$ sudo python setup.py clean

running clean
Traceback (most recent call last):
  File "setup.py", line 67, in <module>
    'clean': CleanCommand,
  File "/home/jc/.local/lib/python2.7/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()

I get error like that. could you explain more detail how to use this code?

thank you

Example of using Inception v3

Are you able to provide a sample on the best practice for feeding an image into a Inception3 model.

At the moment I have:

t_input = tf.placeholder(tf.uint8, name='input')
image = Image.open(imageFileName)
im = np.array(image)[:, :, 0:3] 
tf.import_graph_def(graph_def, {'DecodeJpeg:0': t_input})

This works but in your sample for Inception5h you resize the image at the same time, due to the different input types between v3 and v5 I am not sure how to get the equivilent behaviour

ImportError: No module named 'vgg'

hi, why is the error?

Some points unclear and references missing

Hey,

There are a few points which seem to be unclear for understanding. Maybe you could explain a few things briefly or add some references to the README.md file. That would be helpful, thanks.

Cheers.

AttributeError: module 'tensorflow.python.ops.gen_nn_ops' has no attribute '_relu_grad'

Currently, I'm using tensorflow 1.8. When I run the example code with the deconv_visualization() function.

It returns error:

  File "/Yang/project/detection/code/tf18/third_party/tf_cnnvis/tf_cnnvis/tf_cnnvis.py", line 43, in _GuidedReluGrad
    return tf.where(0. < grad, gen_nn_ops._relu_grad(grad, op.outputs[0]), tf.zeros_like(grad))
AttributeError: module 'tensorflow.python.ops.gen_nn_ops' has no attribute '_relu_grad'

Do you have any idea about that. I'm really appreciate that!

can't utilize GPU to accelerate computation.

Hi,
I tried to visualize the InceptionV4 model layers and fed a MRI image to it, everything works well except that the GPU seems did not involved in computing. TensorFlow do allocate graphic memory for the process but the GPU utilization rate is at 0%. Most of time only one CPU core is working. How can I utilize GPU to accelerate? below are my codes.

`
import tensorflow as tf
from tf_cnnvis import *
from nets.inception_v4 import inception_v4_base, inception_v4_arg_scope
import matplotlib.image as mpimg
import numpy as np

slim = tf.contrib.slim

if name == 'main':
X = tf.placeholder(tf.float32, [None, 160, 160, 3])
img = mpimg.imread('data/image.png')
img = img[42:202, 3:163]
img = np.stack([img, img, img], axis=2)
img = np.reshape(img, [1, 160, 160, 3])

with slim.arg_scope(inception_v4_arg_scope()):
    net_out = inception_v4_base(inputs=X)

t_vars = tf.trainable_variables()
IV4_vars = [var for var in t_vars if var.name.startswith('InceptionV4')]

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    saver = tf.train.Saver(var_list=IV4_vars)
    saver.restore(sess,
                  'checkpoint/inception_v4_2016_09_09/inception_v4.ckpt')

    _ = deconv_visualization(
        sess_graph_path=sess,
        value_feed_dict={X: img},
        layers=['r', 'p', 'c'],
        path_logdir='summary/cnnvis/log',
        path_outdir='summary/cnnvis/out')

`

Thanks for the excellent work : )

Usage for custom CNN

Hi,
Thanks for you work. I tried your example: example/tf_cnnvis_Example1;ipynb, it works well with AlexNet.

So I adapted this function activation_visualization with my own CNN architecture defined like this:

Conv1_1+relu+conv1_2+relu+pool1+conv2_1+relu+conv2_2+relu+pool2+fc1

The activation map was successfully saved for conv1_1 and conv1_2, but it gave error for conv2_1 at this function:
def _activation(graph, sess, op_tensor, feed_dict): with graph.as_default() as g: with sess.as_default() as sess: act = sess.run(op_tensor, feed_dict = feed_dict) return act

The error from this: act = sess.run(op_tensor, feed_dict = feed_dict), I think it is the problem of the shapes for op_tensor (?, 112, 112, 64) and feed_dict (?, 224, 224, 3)..

The error is: InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder_2' with dtype float

As my CNN is slightly different with AlexNet, I did not use any local response normalization. So my question is, is your visualizer works for general CNN model or a specific architecture?

Thanks in advance

Look forward to your response.

Hao

Have some question about generate folder

hi
I run successfully and generate some folders but I don't understand what is conv2d-conv2d_7

Quickfix for Python 3.x setup.py UnicodeDecodeError

In setup.py, change function read() to the following:

def read(fname):
	if six.PY2:
		return open(os.path.join(os.path.dirname(__file__), fname)).read()
	else:
		return open(os.path.join(os.path.dirname(__file__), fname), encoding='latin1').read()

This will fix the issue if some users in Python 3.X encounter UnicodeDecodeError during the installation 👍

The output is inconsistent, when output multiple times from the same image processing.

This is observed when I wrote a bug in my code. I happened to output the deconvolution result for the same image multiple times within a loop:

it looks like this:

for i in range(len(testImages)):
      tfcv.deconv_visualization(graph_or_path = tf.get_default_graph(), value_feed_dict = {x : testImages[0:1], y_: testLabels[0:1], keep_prob: 1.0}, input_tensor = xInput, layers = ["r"], path_logdir=logDir + "/Deconvolution/", path_outdir=saveDir + "/Deconvolution/")

I should have corrected the bug.
However, when I looked into the results. I found the result is not the same across the repeating times.

This makes me confused.
I'm expecting there is a bug in the implementation of tf_cnnvis.

Cannot assign a device for operation error

Hi,

I was running activation_visualization on my graph checkpoint, and it throws this error. Do you know why?

InvalidArgumentError: Cannot assign a device for operation 'fake_var_3': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]. Make sure the device specification refers to a valid device.
[[Node: fake_var_3 = VariableV2container="", dtype=DT_FLOAT, shape=[1], shared_name="", _device="/device:GPU:0"]]

Maybe allow_soft_placement should be set to True when creating the TF session?

Running the tf_connvis_example1.ipynb ，then having a error

hi ,when i running the example,generating a error ：

OSError: [Errno 22] Invalid argument: './Out/AlexNet\conv1\image_2017-12-22_10:16:21\activations\grid_activation.png'

deep dream example images

Thanks for your work! I wonder if you can share the exact settings to generate samples on the Figure 2 like Cauliflower. I didn't manage to get the same result just by changing the class number.

How to generate bigger images

On your homepage you have large images but when i run the code the grid is 8x8 = 254px*254px, how can i generate larger images?

Also does it have to generate the grid, can i get it to generate a file for each image?

AttributeError: 'str' object has no attribute 'name

Hi,
I am trying to use your code to visualize the max activation of different layers in my deep neural network but I keep receiving the following error which I assume has something to do with the way the names of the layers are defined.
the error:

Traceback (most recent call last):

File "", line 1, in
runfile('/media/asgharpn/daten2017-03/Bone_Machine_learning/ML_temp/Deeploc_test/Visualization_max_activation_00.py', wdir='/media/asgharpn/daten2017-03/Bone_Machine_learning/ML_temp/Deeploc_test')

File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 699, in runfile
execfile(filename, namespace)

File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 81, in execfile
builtins.execfile(filename, *where)

File "/media/asgharpn/daten2017-03/Bone_Machine_learning/ML_temp/Deeploc_test/Visualization_max_activation_00.py", line 164, in
path_logdir='./Log', path_outdir='./Output')

File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 410, in deconv_visualization
path_logdir = path_logdir, path_outdir = path_outdir)

File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 166, in _get_visualization
is_success = _visualization_by_layer_name(g, value_feed_dict, input_tensor, layer, method, path_logdir, path_outdir)

File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 270, in _visualization_by_layer_name
parsed_tensors = parse_tensors_dict(graph, layer_name, value_feed_dict)

File "build/bdist.linux-x86_64/egg/tf_cnnvis/utils.py", line 79, in parse_tensors_dict
tmp = get_tensor(graph = g, name = key_op.name)

AttributeError: 'str' object has no attribute 'name'

Here is the architecure of my trained network with 13 layers (8 convolutional, 3 relu and 2 fully connected layers):

n_feat_c1 = 8
n_feat_c2 = 8
n_feat_c3 = 16
n_feat_c4 = 16
n_feat_c5 = 32
n_feat_c6 = 32
n_feat_c7 = 64
n_feat_c8 = 64
n_feat_fc1= 128
n_feat_fc2= 128
numClasses = 4

def DeepLocModel(input_images,is_training):

conv1 = nn_layers.conv_layer(input_images, 3, 3, 1, n_feat_c1, 1, 'conv_1',is_training=is_training)
print conv1.get_shape
conv2 = nn_layers.conv_layer(conv1, 3, 3, n_feat_c1, n_feat_c2, 1, 'conv_2', is_training=is_training)
print conv2.get_shape
pool1 = nn_layers.pool2_layer(conv2, 'pool1')
print pool1.get_shape
conv3 = nn_layers.conv_layer(pool1, 3, 3, n_feat_c2, n_feat_c3, 1, 'conv_3', is_training=is_training)
print conv3.get_shape
conv4 = nn_layers.conv_layer(conv3, 3, 3, n_feat_c3, n_feat_c4, 1, 'conv_4', is_training=is_training)
print conv4.get_shape
pool2 = nn_layers.pool2_layer(conv4, 'pool2')
print pool2.get_shape
conv5 = nn_layers.conv_layer(pool2, 3, 3, n_feat_c4, n_feat_c5, 1, 'conv_5', is_training=is_training)
print conv5.get_shape
conv6 = nn_layers.conv_layer(conv5, 3, 3, n_feat_c5, n_feat_c6, 1, 'conv_6', is_training=is_training)
print conv6.get_shape
conv7 = nn_layers.conv_layer(conv6, 3, 3, n_feat_c6, n_feat_c7, 1, 'conv_7', is_training=is_training)
print conv7.get_shape
conv8 = nn_layers.conv_layer(conv7, 3, 3, n_feat_c7, n_feat_c8, 1, 'conv_8', is_training=is_training)
print conv8.get_shape
pool3 = nn_layers.pool2_layer(conv8, 'pool3')
print pool3.get_shape
size_last_pool= pool3.shape
size_last_pooled_image = int(size_last_pool[1]*size_last_pool[2]*size_last_pool[3])
pool3_flat = tf.reshape(pool3, [-1, size_last_pooled_image])
print pool3_flat.get_shape
fc_1 = nn_layers.nn_layer(pool3_flat, size_last_pooled_image, n_feat_fc1, 'fc_1', act=tf.nn.relu, is_training=is_training)
fc_2 = nn_layers.nn_layer(fc_1, n_feat_fc1, n_feat_fc2, 'fc_2', act=tf.nn.relu,is_training=is_training)
logit = nn_layers.nn_layer(fc_2, n_feat_fc2, numClasses, 'final_layer', act=None, is_training=is_training)

return logit

and finally here is the code I use to visualize the activations using tf_cnnviz:

sess = tf.Session()
sess.run(tf.global_variables_initializer(),{is_training:False})

load model checkpoint

saver = tf.train.Saver(tf.global_variables())
saver.restore(sess, locNetCkpt)

Loading labels from test data set

dataBaseDir = '/media/asgharpn/daten2017-03/Bone_Machine_learning/Learning_dataset/projected_augmented_not_squared_yz_test/'
trainHdf5 = dataBaseDir+'bone_projected_valid_set.hdf5'
f = h5py.File(trainHdf5,'r')
labels = f['Index1']
dset = f['data1']

cropSize = 733
batchSize = dset.shape[0]-1
stretchLow = 0.1 # stretch channels lower percentile
stretchHigh = 99.9 # stretch channels upper percentile

imSize_x = 733
imSize_z = 161
numClasses = 4
numChan = 1

data = dataClass.Data(trainHdf5,['data','Index'],batchSize)
batch = data.getBatch()
loc = tf.Graph()
with loc.as_default():
loc_saver = tf.train.import_meta_graph(locNetCkpt+'.meta')
locSession = tf.Session(graph=loc)
loc_saver.restore(locSession, locNetCkpt)

pred_loc = loc.get_tensor_by_name(u'softmax:0')
input_loc = loc.get_tensor_by_name(u'input:0')
is_training_loc = loc.get_tensor_by_name(u'is_training:0')
processedBatch=procIm.preProcessImages(batch['data'],
imSize_x,imSize_z,cropSize,numChan,
rescale=False,stretch=False,
means=None,stds=None,
stretchLow=stretchLow,stretchHigh=stretchHigh,
jitter=False,randTransform=False)

images = processedBatch[10][:]
is_training_loc = loc.get_tensor_by_name(u'is_training:0')
input_loc = loc.get_tensor_by_name(u'input:0')
feed_dict={input_loc: images, is_training_loc: False}

deconv visualization

layers = ["c"]
total_time = 0
x = tf.placeholder(tf.float32, [None, 118013])
y_ = tf.placeholder(tf.float32, [None, 4])
keep_prob = tf.placeholder(tf.float32)

x_image = tf.reshape(x, [-1, 733, 161, 1])
start = time.time()

feed_dict = {x:batch['data'][1:2], y_: batch['Index'][1:2], keep_prob: 1.0}
is_success = deconv_visualization(sess_graph_path = sess, value_feed_dict = batch,
input_tensor=x_image, layers=layers,
path_logdir='./Log', path_outdir='./Output')
start = time.time() - start
print("Total Time = %f" % (start))

sess.close()

Could you please help me with this error?

Is it possible to only visualize one layer?

by using the deconv_visualization() function, it generates deconv images for each layers by default. Is it possible to only generate images for one single layer? Thank you.

visualization of concatenation operation

thank you for the handy visualization tool
in my code, i have a concatenation operation (tf.concat) defined in the network, but the tool is unable to visualize any layer after the concatenation layer. would you please help to take a look?

Fail to create frozen graph.

I'm trying to do visualization on custom pre trained model and I met with the problem after I tried to create frozen graph like this:

od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
    serialized_graph = fid.read()
    od_graph_def.ParseFromString(serialized_graph)
tf.import_graph_def(od_graph_def, name='')

then I tried to do the activation visualization:

layers = ['r', 'p', 'c']

start = time.time()
temp = tf.get_default_graph()
print(temp)
X = tf.placeholder(np.uint8, shape = [1, 370, 1224, 3])
# with detection_graph.as_default():
# with sess_graph_path = None, the default Session will be used for visualization.\
is_success = activation_visualization(sess_graph_path = tf.get_default_graph(), value_feed_dict = {X : im}, 
                                          layers=layers, path_logdir=os.path.join("Log","Kitti"), 
                                          path_outdir=os.path.join("Output","Kitti"))
start = time.time() - start
print("Total Time = %f" % (start))

Then I got the error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-82e3351f0452> in <module>()
     10 is_success = activation_visualization(sess_graph_path = tf.get_default_graph(), value_feed_dict = {X : im}, 
     11                                           layers=layers, path_logdir=os.path.join("Log","Kitti"),
---> 12                                           path_outdir=os.path.join("Output","Kitti"))
     13 start = time.time() - start
     14 print("Total Time = %f" % (start))

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tf_cnnvis-1.0.0-py2.7.egg/tf_cnnvis/tf_cnnvis.pyc in activation_visualization(sess_graph_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir)
    404 def activation_visualization(sess_graph_path, value_feed_dict, input_tensor = None,  layers = 'r', path_logdir = './Log', path_outdir = "./Output"):
    405     is_success = _get_visualization(sess_graph_path, value_feed_dict, input_tensor = input_tensor, layers = layers, method = "act",
--> 406         path_logdir = path_logdir, path_outdir = path_outdir)
    407     return is_success
    408 def deconv_visualization(sess_graph_path, value_feed_dict, input_tensor = None,  layers = 'r', path_logdir = './Log', path_outdir = "./Output"):

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tf_cnnvis-1.0.0-py2.7.egg/tf_cnnvis/tf_cnnvis.pyc in _get_visualization(sess_graph_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir, method)
    129     # convert all inplicit and explicit sess input cases to a PATH
    130     if isinstance(sess_graph_path, tf.Graph):
--> 131         PATH = _save_model(sess_graph_path)
    132     elif isinstance(sess_graph_path, tf.Session):
    133         PATH = _save_model(sess_graph_path)

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tf_cnnvis-1.0.0-py2.7.egg/tf_cnnvis/tf_cnnvis.pyc in _save_model(graph_or_sess)
     63         for op in ops:
     64             if 'variable' in op.type.lower():
---> 65                 raise ValueError('Please input a frozen graph (no variables). Or pass in the session object.')
     66 
     67         with graph_or_sess.as_default():

ValueError: Please input a frozen graph (no variables). Or pass in the session object.

I'm not sure if I wrongly created the frozen graph. Please help me if I did it wrongly. Many thanks!

how to use other CNN architecture(for example VGG,GOOGLENET...) to implement visualization?

Hi~
very thankful for you nice work! I am kind of new to tensorflow, I have run tf_cnnvis_Example1.ipynb successfully and I am wondering how to replace AlexNet with some other pretrained CNN architecture (for example ,VGG)
hope to get your response~
appreciate it a lot

AttributeError: 'NoneType' object has no attribute 'outer_context'

Hi,
I am trying to visualize inception_resent_v2 layers.

    is_loaded = tf_cnnvis.activation_visualization(graph_or_path= detection_graph, 
                                          value_feed_dict={image_tensor: image_np_expanded},
                                           layers= 'p', 
                                           path_logdir= data_dir + "results/act",
                                           path_outdir= data_dir + "results/act")
            
      is_loaded = tf_cnnvis.deconv_visualization(graph_or_path= detection_graph, 
                                           value_feed_dict={image_tensor: image_np_expanded},
                                           layers= 'p', 
                                           path_logdir= data_dir + "results/rec",
                                           path_outdir= data_dir + "results/rec")

And that is how I load the Graph;

              # Load a (frozen) Tensorflow model into memory
              detection_graph = tf.Graph()
              with detection_graph.as_default():
                 od_graph_def = tf.GraphDef()
                 with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
                      serialized_graph = fid.read()
                      od_graph_def.ParseFromString(serialized_graph)
                      tf.import_graph_def(od_graph_def, name='')

I use tf_cnnvis twice, first one to visualize the activation layers, and second one for deconstruction.

The first one is working successfully, but when I try to deconstruction I get the following error;

AttributeError: 'NoneType' object has no attribute 'outer_context'

any idea what is wrong?

Regards

AttributeError when running example 1

I got this error when running example 1:

     41         @ops.RegisterGradient("GuidedRelu")
     42         def _GuidedReluGrad(op, grad):
---> 43             return tf.where(0. < grad, gen_nn_ops._relu_grad(grad, op.outputs[0]), tf.zeros_like(grad))
     44 
     45         is_Registered = True

AttributeError: 'module' object has no attribute '_relu_grad'

Any idea why that happened? Thanks!

Why I can not find any code about deconvnet, unpooling, recrification introduced in 'Visualizing and Understanding Convolutional Networks'?

Hi, thinks for your pretty work. I read the code of this project, but I have a problem. Can you give me an explain? In paper 'Visualizing and Understanding Convolutional Networks', an approach based on deconvnet, unpooling, recrification are introduced. In 'Readme' you see you infer 'Visualizing and Understanding Convolutional Networks', but I can not find any code about deconvnet, unpooling, recrification operations. I find the output is just the gradient about layer-out tensor and X in function '_deconvolution' in tf_cnnvis.py. Is this relevant to deconvnet, unpooling, recrification introduced in 'Visualizing and Understanding Convolutional Networks'?

No attribute _relu_grad

Hello there, I've tried to run the examples using tensorflow v1.8.0 on ubuntu 16.04 and macOS High Sierra, python v3, but I get the following error:

example_1, method 'deconv_visualization': AttributeError: module 'tensorflow.python.ops.gen_nn_ops' has no attribute '_relu_grad'

I investigated a bit and it seems that gen_nn_ops is generated by basel. I think the error comes from a difference in versions. Should we try to update the code to the latest versions?

Questions on Example 3, sessions and graphs

I the 3 examples, the activation functions are taking graphs as input and producing visualizations as output. As a graph only stores the structure of the network and not the actual weight values, how can the visualization functions have access to these values? To my understanding in example 1 the the default values for the graph nodes are visualized, in example 2 there seems to be a solvable problem, but how about Example 3?
I see a tmp_model folder which could hold the data needed to restore the variable values, but where is this data in tmp_model generated?
Thank you for your help and patience

some question about this code

creating placeholders to pass featuremaps and

		# creating gradient ops
		featuremap = [tf.placeholder(tf.int32) for i in range(config["N"])]
		reconstruct = [tf.gradients(tf.transpose(tf.transpose(op_tensor)[featuremap[i]]), X)[0] for i in range(config["N"])]

		# Execute the gradient operations in batches of 'n'
		for i in range(0, tensor_shape[-1], config["N"]):
			c = 0
			for j in range(config["N"]):
				if (i + j) < tensor_shape[-1]:
					feed_dict[featuremap[j]] = i + j
					c += 1
			if c > 0:
				out.extend(sess.run(reconstruct[:c], feed_dict = feed_dict))

I do not understand this code, why it is, i can not see deconv or Unpooling

Running deep dream gives error "invalid argument: computed output size would be negative"

As per the title, when i run the sample 3 using inception5 and the image of the wolf and use deepdream_visualization i end up getting this error over and over again.

Any ideas as to why?

Deepdream for Mobilenet V2 COCO model

I'm trying deepdream_visualization on ssdlite_mobilenet_v2_coco model by modifying the tf_cnnvis_Example3.ipynb. I have set the input layer to FeatureExtractor/MobilenetV2/MobilenetV2/input and visualisation layer to import/FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/batchnorm/add_1

Given these layers, deepdream_visualization generated uninterpretable image as shown below

I'm not sure if I have chosen the correct layers for input and visualization. Any pointers on this would be very useful

Thanks

How to modify deepdream for single channel image?

The deep dream visualisation seems to be fixed to 3-channel images. How to modify for 1 channel images?

Object detection net(eg:faster_rcnn_resnet101) not worked with deconv_visualization

Object detection net(eg:faster_rcnn_resnet101) not worked with deconv_visualization.
With activation_visualization works well.
Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-9aba9f0fced7> in <module>()
      3                                      layers=layers,
      4                                      path_logdir=os.path.join("Log","Inception5"),
----> 5                                      path_outdir=os.path.join("Output","Inception5"))

/notebooks/workspace/github/tf_cnnvis/tf_cnnvis/tf_cnnvis.py in deconv_visualization(sess_graph_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir)
    408 def deconv_visualization(sess_graph_path, value_feed_dict, input_tensor = None,  layers = 'r', path_logdir = './Log', path_outdir = "./Output"):
    409     is_success = _get_visualization(sess_graph_path, value_feed_dict, input_tensor = input_tensor, layers = layers, method = "deconv",
--> 410         path_logdir = path_logdir, path_outdir = path_outdir)
    411     return is_success
    412 

/notebooks/workspace/github/tf_cnnvis/tf_cnnvis/tf_cnnvis.py in _get_visualization(sess_graph_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir, method)
    167                 elif layer != None and layer.lower() in dict_layer.keys():
    168                     layer_type = dict_layer[layer.lower()]
--> 169                     is_success = _visualization_by_layer_type(g, value_feed_dict, input_tensor, layer_type, method, path_logdir, path_outdir)
    170                 else:
    171                     print("Skipping %s . %s is not valid layer name or layer type" % (layer, layer))

/notebooks/workspace/github/tf_cnnvis/tf_cnnvis/tf_cnnvis.py in _visualization_by_layer_type(graph, value_feed_dict, input_tensor, layer_type, method, path_logdir, path_outdir)
    225 
    226     for layer in layers:
--> 227         is_success = _visualization_by_layer_name(graph, value_feed_dict, input_tensor, layer, method, path_logdir, path_outdir)
    228     return is_success
    229 

/notebooks/workspace/github/tf_cnnvis/tf_cnnvis/tf_cnnvis.py in _visualization_by_layer_name(graph, value_feed_dict, input_tensor, layer_name, method, path_logdir, path_outdir)
    289         elif method == "deconv":
    290             # deconvolution
--> 291             results = _deconvolution(graph, sess, op_tensor, X, feed_dict)
    292         elif method == "deepdream":
    293             # deepdream

/notebooks/workspace/github/tf_cnnvis/tf_cnnvis/tf_cnnvis.py in _deconvolution(graph, sess, op_tensor, X, feed_dict)
    335                         c += 1
    336                 if c > 0:
--> 337                     out.extend(sess.run(reconstruct[:c], feed_dict = feed_dict))
    338     return out
    339 def _deepdream(graph, sess, op_tensor, X, feed_dict, layer, path_outdir, path_logdir):

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    898     try:
    899       result = self._run(None, fetches, feed_dict, options_ptr,
--> 900                          run_metadata_ptr)
    901       if run_metadata:
    902         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
   1118     # Create a fetch handler to take care of the structure of fetches.
   1119     fetch_handler = _FetchHandler(
-> 1120         self._graph, fetches, feed_dict_tensor, feed_handles=feed_handles)
   1121 
   1122     # Run request and get response.

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in __init__(self, graph, fetches, feeds, feed_handles)
    425     """
    426     with graph.as_default():
--> 427       self._fetch_mapper = _FetchMapper.for_fetch(fetches)
    428     self._fetches = []
    429     self._targets = []

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in for_fetch(fetch)
    243     elif isinstance(fetch, (list, tuple)):
    244       # NOTE(touts): This is also the code path for namedtuples.
--> 245       return _ListFetchMapper(fetch)
    246     elif isinstance(fetch, dict):
    247       return _DictFetchMapper(fetch)

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in __init__(self, fetches)
    350     """
    351     self._fetch_type = type(fetches)
--> 352     self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
    353     self._unique_fetches, self._value_indices = _uniquify_fetches(self._mappers)
    354 

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in <listcomp>(.0)
    350     """
    351     self._fetch_type = type(fetches)
--> 352     self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
    353     self._unique_fetches, self._value_indices = _uniquify_fetches(self._mappers)
    354 

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in for_fetch(fetch)
    240     if fetch is None:
    241       raise TypeError('Fetch argument %r has invalid type %r' % (fetch,
--> 242                                                                  type(fetch)))
    243     elif isinstance(fetch, (list, tuple)):
    244       # NOTE(touts): This is also the code path for namedtuples.

TypeError: Fetch argument None has invalid type <class 'NoneType'>

Addition:

Variable reconstruct (defined at line 327 in tf_cnnvis.py ) is [None, None, None, None, None, None, None, None]. TensorFlow version is 1.9. When I use TensorFlow 1.4, I got same Error as #26

No output to log or output directory

Hello, I'm rather new to all this so forgive me if this is a daft question. However, i just can't get the examples to work with my data. I've basically followed the "Training" section of https://www.tensorflow.org/tutorials/image_retraining and have had some good results from the resulting model. However, i want to visualise them and your library seems perfect for the job!

What am i doing wrong in the code below? I get no errors and 'is_success' is True so it appears to have worked?

import os
import sys
import time
import copy
import h5py
import numpy as np

from tf_cnnvis import *

import tensorflow as tf
from scipy.misc import imread, imresize

t_input = tf.placeholder(np.float32, name='Placeholder') # define the input tensor

graph = tf.Graph()
graph_def = tf.GraphDef()

with open('./path/to/my/graph.pb', "rb") as f:
  graph_def.ParseFromString(f.read())

with graph.as_default():
  tf.import_graph_def(graph_def)

# reading sample image
im = np.expand_dims(imread(os.path.join("./", "FILENAME.JPG")), axis = 0)

tensor_names = [t.name for op in graph.get_operations() for t in op.values()]
print(tensor_names[0])
print(tensor_names[len(tensor_names) - 1])

start = time.time()
# api call

is_success  = activation_visualization(
                                            sess_graph_path=tf.get_default_graph(),
                                            value_feed_dict = {t_input : im}, 
                                            input_tensor=tensor_names[len(tensor_names) - 1],
                                            layers=[ 'r', 'p', 'c'], 
                                            path_outdir="./output",
                                            path_logdir="./log"
                                         )
start = time.time() - start
print("Total Time = %f" % (start))
print(is_success)

The main things i don't seem to understand is the value_feed_dict, the input_tensor and the t_input / im variables.

I'm sure i'm doing something daft but would be trivial for someone in the know to spot what is wrong.

Thank you!

visualization for semantic segmentation task is possible?

Hi,

i have mostly seen people use visualization for image recognition models. Does it make sense and is it possible to use your code for visualizing FCN network layers?

installation platform

Hello,
Which installation platform is suit for,Linus or Windows?
Thank you.

how to use this on audio data?

I am computing mfcc for audio file, andtrain a cnn. How do I visualize what the network is learning at different layers?

3D CNN visualization

Hi,

I also have a question, do you plan to implement the 3D version to visualize the activation map and decent???

Thanks in advance

Fetch argument None has invalid type <class 'NoneType'>

Hi. I'm having a lot of trouble trying to get your function to work. I'm getting an error now that looks like it is coming from your deconvolution functions. Could you please let me know what you think the solution is?

Code I used:

is_loaded=tf_cnnvis.deconv_visualization(graph_or_path=tf.get_default_graph(),
                                         value_feed_dict={x_pl:x_batch, y_gt:targets_batch, is_training:False, valid_eval_accs:1000.0, valid_xent:1000.0},
                                         layers='r',
                                         path_logdir="c:/p17/logs/tf_cnnvis/flowers/1/logs",
                                         path_outdir="c:/p17/logs/tf_cnnvis/flowers/1/out")

Thanks.

INFO:tensorflow:Restoring parameters from model\tmp-model
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-41761d870143> in <module>()
      3                                          layers='r',
      4                                          path_logdir="c:/p17/logs/tf_cnnvis/flowers/1/logs",
----> 5                                          path_outdir="c:/p17/logs/tf_cnnvis/flowers/1/out")

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tf_cnnvis-1.0.0-py3.5.egg\tf_cnnvis\tf_cnnvis.py in deconv_visualization(graph_or_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir)
    381 def deconv_visualization(graph_or_path, value_feed_dict, input_tensor = None, layers = 'r', path_logdir = './Log', path_outdir = "./Output"):
    382 	is_success = _get_visualization(graph_or_path, value_feed_dict, input_tensor = input_tensor, layers = layers, method = "deconv", 
--> 383 		path_logdir = path_logdir, path_outdir = path_outdir)
    384         return is_success
    385 def deepdream_visualization(graph_or_path, value_feed_dict, layer, classes, input_tensor = None, path_logdir = './Log', path_outdir = "./Output"):

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tf_cnnvis-1.0.0-py3.5.egg\tf_cnnvis\tf_cnnvis.py in _get_visualization(graph_or_path, value_feed_dict, input_tensor, layers, path_logdir, path_outdir, method)
    149                         elif layers != None and layers.lower() in dict_layer.keys():
    150                                 layer_type = dict_layer[layers.lower()]
--> 151                                 is_success = _visualization_by_layer_type(g, value_feed_dict, input_tensor, layer_type, method, path_logdir, path_outdir)
    152                         else:
    153                                 is_success = False

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tf_cnnvis-1.0.0-py3.5.egg\tf_cnnvis\tf_cnnvis.py in _visualization_by_layer_type(graph, value_feed_dict, input_tensor, layer_type, method, path_logdir, path_outdir)
    202 
    203         for layer in layers:
--> 204                 is_success = _visualization_by_layer_name(graph, value_feed_dict, input_tensor, layer, method, path_logdir, path_outdir)
    205         return is_success
    206 

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tf_cnnvis-1.0.0-py3.5.egg\tf_cnnvis\tf_cnnvis.py in _visualization_by_layer_name(graph, value_feed_dict, input_tensor, layer_name, method, path_logdir, path_outdir)
    263                         elif method == "deconv":
    264                                 # deconvolution
--> 265                                 results = _deconvolution(graph, sess, op_tensor, X, feed_dict)
    266                         elif method == "deepdream":
    267                                 # deepdream

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tf_cnnvis-1.0.0-py3.5.egg\tf_cnnvis\tf_cnnvis.py in _deconvolution(graph, sess, op_tensor, X, feed_dict)
    310                                                 c += 1
    311                                 if c > 0:
--> 312                                         out.extend(sess.run(reconstruct[:c], feed_dict = feed_dict))
    313         return out
    314 def _deepdream(graph, sess, op_tensor, X, feed_dict, layer, path_outdir, path_logdir):

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata)
    787     try:
    788       result = self._run(None, fetches, feed_dict, options_ptr,
--> 789                          run_metadata_ptr)
    790       if run_metadata:
    791         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    982     # Create a fetch handler to take care of the structure of fetches.
    983     fetch_handler = _FetchHandler(
--> 984         self._graph, fetches, feed_dict_string, feed_handles=feed_handles)
    985 
    986     # Run request and get response.

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in __init__(self, graph, fetches, feeds, feed_handles)
    408     """
    409     with graph.as_default():
--> 410       self._fetch_mapper = _FetchMapper.for_fetch(fetches)
    411     self._fetches = []
    412     self._targets = []

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in for_fetch(fetch)
    228     elif isinstance(fetch, (list, tuple)):
    229       # NOTE(touts): This is also the code path for namedtuples.
--> 230       return _ListFetchMapper(fetch)
    231     elif isinstance(fetch, dict):
    232       return _DictFetchMapper(fetch)

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in __init__(self, fetches)
    335     """
    336     self._fetch_type = type(fetches)
--> 337     self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
    338     self._unique_fetches, self._value_indices = _uniquify_fetches(self._mappers)
    339 

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in <listcomp>(.0)
    335     """
    336     self._fetch_type = type(fetches)
--> 337     self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
    338     self._unique_fetches, self._value_indices = _uniquify_fetches(self._mappers)
    339 

C:\Users\username\Anaconda3\envs\tfGPU\lib\site-packages\tensorflow\python\client\session.py in for_fetch(fetch)
    225     if fetch is None:
    226       raise TypeError('Fetch argument %r has invalid type %r' %
--> 227                       (fetch, type(fetch)))
    228     elif isinstance(fetch, (list, tuple)):
    229       # NOTE(touts): This is also the code path for namedtuples.

TypeError: Fetch argument None has invalid type <class 'NoneType'>

is this tool support vis fc layer?

hello i want to know whether this tool support fc layer or not, i find it only support pooling,rely,and cone layer..

About conv2d and conv2_1

Hello, thank you for your valuable work.
I have a doubt .
I have run the example 1 and get the visiualization on the tensorboard.
What is the difference between the reconstructed feature conv2_1(or conv2_2) with the conv2d(conv2d_1--7) ?
I want to know where the feature conv2d(1--7) are from.
Look forward to your reply!

please add an option to the save image name

I read the codes in utils.py and I found that all the output images from the same layer have the same file name (grid_activation.png). If I test more one image, the previous image would be replaced with later image. Could you please add an option to change the image name. Thank you very much!

for i in range(len(grid_activations)):
    time_stamp = time.time()
    time_stamp = datetime.datetime.fromtimestamp(time_stamp).strftime('%Y-%m-%d_%H-%M-%S')

    grid_activation_path = os.path.join(path_out, "activations")
    is_success = make_dir(grid_activation_path)
    imsave(os.path.join(grid_activation_path, "grid_activation.png"), grid_activations[i][0,:,:,0], format = "png")

Can someone give me steps to try out this toolbox on my own model?

I have a trained tensorflow model that I want to visualize. Can someone tell me how to do so? Which files to edit. I have seen the three examples given but I can't infer how to do so on my model.
If any details are required, kindly ask.

Examples do not replicate description results?

Hello!

In description:
"Figure 1: Original image and the reconstructed versions from maxpool layer 1,2 and 3 of Alexnet generated using tf_cnnvis."

After running tf_cnnvis_Example1.ipynb got deconvolutions folders with grided images, with
non of the picture from the grid seems to be a deconvolution image as in description.

The most look-like image from maxpool_3/deconvolution is:

The most look-like image from maxpool_4/deconvolution is:

Is there a way get the same picture as in description with your tool?

example2 Session and vaiabale re-initialization

I am using this project and am going trough example2 on MNIST data to get a better grasp on how tensorflow and cnn visualization works. I apologize in advance if I am missing some trivial aspect and my question is not relevant.

To my understanding, block 3 in example2 trains a cnn on MNIST data to learn weights and block 4 uses tf_cnnvis functions to inspect what the network has learned. The training is performed in a with tf.Session() as sess: block, while a new Session and subsequent tf.global_variables_initializer() call is issued in _visualization_by_layer_name within deconv_visualization.

Would this not wipe off the weights learned in training?

Can we have a "MNIST" visualization tutorial ?

I am beginner with tensorflow and CNN. So can we have a tutorial about how to use it ?

deepdream visualization on convolution layer

I am not so sure how to use the deepdream api to visualisation on a particular convolution layer (say after a max pooling), the classes option is only for final softmax output?

Deconvolution seems not been used in the project?

Hi, thanks for your valuable work! I have learned a lot from your codes.
I'm trying to implement the method described in "Visualizing and Understanding Convolutional Networks" by Matthew D. and they seemed to have used the deconvolution method. But I couldn't find the deconvolution method in your codes even in the _deconvolution function. The reconstruction seems to be gotten by tf.gradients() rather than the tf.nn.conv2d_transpose() which is the official deconvolution method.
reconstruct = [tf.gradients(tf.transpose(tf.transpose(op_tensor)[featuremap[i]]), X)[0] for i in range(config["N"])]
Although the reconstruction by your codes looks like to the results of that paper, I'm not sure whether or not you are using the same method. Can you explain a little bit about the differences?
Look forward to your reply!

how to visualize network with BN?

my network was trained from scratch, so i use some BN layer.Should I modify the BN layer gradient?

why I always meet this problem?

Here's the error I'm getting

ValueError: Cannot feed value of shape (1, 224, 224, 3) for Tensor u'Placeholder:0', which has shape '(2, 224, 224, 3)'

Here's the code for reproducing the error:

---

images = tf.placeholder("float", [2, 224, 224, 3]) batch1 = img1.reshape((1, 224, 224, 3)) batch2 = img2.reshape((1, 224, 224, 3)) batch = np.concatenate((batch1, batch2), 0) feed_dict = {images: batch} is_success = activation_visualization(graph_or_path=tf.get_default_graph(),value_feed_dict={X:batch},layers='r')

---

Network is vgg16

Please add an option to output the numerical values

Hi,
This visualization tool is really a big help. But I have encountered some issue when using this tool.
Currently, this tool only outputs the images and everything seems to be fine if the input to the neural network is a real image. But when the input is not a real image(assume that the input is a tensor of size NNC, where C=4 is the number of channels; note that the input is just a feature of 4 channels, not a real image), the output of tf_cnnvis.deconv_visualization will be of little use because the output feature is interpreted as an RGBA image.
Therefore, I propose this issue to add an option to output the numerical values besides the images. Maybe you can add a Boolean flag "output_value" in the function to decide whether the numerical value of the corresponding images will be shown.
Thank you in advance.

Example not working: No Layer with layer name = conv1...

Hi! I got this error when I tried to run the example.. It says:

No Layer with layer name = conv1
No Layer with layer name = conv2_1
No Layer with layer name = conv2_2
No Layer with layer name = conv3
No Layer with layer name = conv4_1
No Layer with layer name = conv4_2
No Layer with layer name = conv5_1
No Layer with layer name = conv5_2
Skipping. Too many featuremap. May cause memory errors.
Skipping. Too many featuremap. May cause memory errors.
No Layer with layer name = MaxPool
No Layer with layer name = MaxPool_1
No Layer with layer name = MaxPool_2
No Layer with layer name = MaxPool_3
No Layer with layer name = MaxPool_4
Total Time = 39.663317

When I tried to use the command:
tf_cnnvis.get_visualization(graph_or_path = tf.get_default_graph(), value_feed_dict = feed_dict, input_tensor=None, layers=['r','p','c]', path_logdir='./Log',
path_outdir='./Output', force=False, n=8)
I tried it in a simple model with only 2 conv layers, 1 max_pool and 2 fc. It doesn't generate any outputs/log files..

Thank you in advance for looking into the problem I'm having.