log0/neural-style-painting

Implementing of the "A Neural Algorithm of Artistic Style"

Arctic Fox

An open source minimal theme for Jekyll, ideal for blogging.

View the demo at http://diezcami.github.io/arctic-fox-theme

Contribute

Please submit open issues for bugs or TBA features that haven't been documented yet! Feel free to submit pull requests for any existing issues.

License

The MIT License (MIT)

Copyright (c) 2015 Camille Diez

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

neural-style-painting's Issues

assert layer_name == expected_layer_name？

art.py, line 161, in _conv2d
W, b = _weights(layer, layer_name)
File "D:/art.py", line 146, in _weights
assert layer_name == expected_layer_name
AssertionError

How can I solve this problem?

Invalid arguments

Traceback (most recent call last):
File "D:\py\neural-style-painting\ac.py", line 274, in
content_image = load_image(CONTENT_IMAGE)
File "D:\py\neural-style-painting\ac.py", line 66, in load_image
image = scipy.misc.imread(path)
AttributeError: module 'scipy.misc' has no attribute 'imread'

Different indexes for W, b and layer_name

Hi! For some reason for me the indexes on the layers are different on https://github.com/log0/neural-style-painting/blob/master/art.py#L144

from

        W = vgg_layers[0][layer][0][0][0][0][0]
        b = vgg_layers[0][layer][0][0][0][0][1]
        layer_name = vgg_layers[0][layer][0][0][-2]

to

        W = vgg_layers[0][layer][0][0][2][0][0]
        b = vgg_layers[0][layer][0][0][2][0][1]
        layer_name = vgg_layers[0][layer][0][0][0][0]

I've donwloaded the model from the link provided: http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat

VGG-19 model

Where should the VGG-19 model set go?

Doubt in Content/Style Loss Functions

Hi, I have . a doubt regarding the loss functions. How is the current value of the genearted image(by current, I mean the updated Variable) compared with the actual content/style image.

def content_loss_func(sess, model):
return _content_loss(sess.run(model['conv4_2']), model['conv4_2'])

In this function, sess.run(model['conv4_2']) is basically sess.run(model['conv4_2'],feed_dict={input_image = content_image})

Shouldn't the second parameter be sess.run(model['conv4_2'],feed_dict={input_image = generated_image})

How does just running model['conv4_2'] pick up the latest value of the tensor ' conv4_2' with input as generated image

Same with the style loss function.

the initial cost is 0

the initial cost is 0 so no optimization happens

the network seems don't work

I replace the content image with

and style image is

After 1000 iterations ,the result is

but in anishathalye implementation , the result is

So, maybe a bug ?

Iteration 0
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 100
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 200
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 300
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 400
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 500
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 600
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 700
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 800
('sum : ', 236020.36)
('cost: ', 0.0)
Iteration 900
('sum : ', 236020.36)
('cost: ', 0.0)

this is the log while ALPHA is 100 BETA is 5

Using Cloud GPU

How can i use floydhub/AWS for implementation of this? My PC would hang everytime i came on the step where it applies gradient descent for reducing the loss function.

Reconstruct FC layers with your *.mat file

Hi,

I would to reconstruct all the layers from the VGG network, with fully-connected layers.
In the *.mat file, the layer with id '37' corresponds to the first FC layer.
Weights and biases have shapes : [7,7,512,4096] and [1,4096].

W, b = _weights(37, 'fc6')
print(W.shape)
print(b.shape)
>>> (7, 7, 512, 4096)
>>>  (4096, 1)

The input tensor is from 'avgpool5' :
Tensor("VGG_3/avgpool_5/AvgPool:0", shape=(1, 7, 7, 512), dtype=float32)

This shape (for W) seems to correspond to an operation that combines "flatten + matmul" .. but I don't find an operation with TensorFlow which can handle those variables ..

After, for the next FC layers, weights and biases have normal shapes :

W, b = _weights(39, 'fc7')
print(W.shape)
print(b.shape)

>>> (1, 1, 4096, 4096)
>>> (4096, 1)

Did you find a solution to reconstruct the whole network ?

I'm creating a workshop about Style Transfer, with TensorFlow.
And I would like to give an example of classification with the whole VGG.

Thanks

AssertionError

When i start art.py i get this error:

python art.py

art.py:147: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future
will perform elementwise comparison
assert layer_name == expected_layer_name
Traceback (most recent call last):
File "art.py", line 277, in
model = load_vgg_model(VGG_MODEL)
File "art.py", line 184, in load_vgg_model
graph['conv1_1'] = _conv2d_relu(graph['input'], 0, 'conv1_1')
File "art.py", line 173, in _conv2d_relu
return _relu(_conv2d(prev_layer, layer, layer_name))
File "art.py", line 162, in _conv2d
W, b = _weights(layer, layer_name)
File "art.py", line 147, in _weights
assert layer_name == expected_layer_name
AssertionError

How to fix it?

Thank you

Missing License

Hi, could you add a License to the code? Right now this is Privative Software and I would like to know if I can make a derivative work from it. Thanks!

More info about software without license on github: https://github.com/blog/1964-open-source-license-usage-on-github-com