progamergov/neural-zoom

Infinite Zoom For Style Transfer

neural-zoom

Neural-Zoom works by cropping the output from a style transfer script, and then using the cropped output as either a content image, or an initialization image for the next frame.

If you are looking for the old version of Neural-Zoom, it can be found here.

Different Crop (Zoom) Values

By changing the crop value, you can control the "speed" at which the zoom will be.

Different X And Y Axis Crop Values

You can control the "zoom" for the x axis and y axis separately.

From the left: "The Scream", and "The Starry Night" + "Picasso Self-Portrait"

Parameters:

In addition to all the parameters from your chosen style transfer script, neural-zoom has it's own parameters:

Basic Options:

• -script: Path to the Lua or Python style transfer script. Currently Neural-Style, Fast-Neural-Style, and Neural-Style-PT are supported.
• -num_frames: The total number of frames to create. Default is 30.

Zoom Options:

• -crop: How much to crop each frame from all sides. Default is 64. If both the -crop_height and -crop_width parameters are greater than zero, this parameter will be ignored.
• -crop_width: How much to crop the left and right sides of each frame.
• -crop_height: How much to crop the top and bottom sides of each frame.

Starting Options:

• -starting_image: Optionally skip creating frame zero, and use your own image instead.
• -start_num: The number to start counting from for frame names. Default is 0.

Output options:

• -num_zeros: How many zeros to use for the number portion of each frame name. Default is 0000.
• -num_mode: Whether to use trailing or leading numbers in frame names. If you set this to 0, leading numbers will be used instead of trailing numbers.
• -output_dir: Name of the output image directory. Default is output_dir.
• -verbose: If this flag is present, then the full set of style transfer parameters used, will be printed for each frame.

Setup

• Something to convert your frames into the desired media format, like FFMPEG, or Imagemagick.

• A supported style transfer project.

You can download Neural-Zoom directly to a style transfer project directory, with:

wget -c https://raw.githubusercontent.com/ProGamerGov/Neural-Zoom/master/neural_zoom.py

Usage

The Neural-Zoom specific parameters will remain the same with each style transfer script, but each script will have it's own different required parameters.

Below are some basic examples of how to use Neural-Zoom (with either Python 2.7 or Python 3):

Neural-Style:

python neural_zoom.py -script neural_style.lua -style_image <image.jpg> -content_image <image.jpg> -model_file models/VGG_ILSVRC_19_layers.caffemodel -proto_file models/VGG_ILSVRC_19_layers_deploy.prototxt

Fast-Neural-Style:

neural_zoom.py -script fast_neural_style.lua -model <model.t7> -input_image <image.jpg>

Neural-Style-PT:

python neural_zoom.py -script neural_style.py -style_image <image.jpg> -content_image <image.jpg> -model_file models/models/vgg19-d01eb7cb.pth

From the left, the style transfer project used is Neural-Style, Fast-Neural-Style, and Neural-Style-PT.

neural-zoom's Issues

Zooms without continuty or detail

The effect doesnt have the same look as the previews in the readme

it looks like every image is diferent and has no sense of zoom or continuity. I used the same images in diferent presets but doesnt seem like it works.

Can't start the script for some odd reason.

'python3' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "neural_zoom.py", line 229, in
main()
File "neural_zoom.py", line 112, in main
first_run(run, params.output_image)
File "neural_zoom.py", line 146, in first_run
first_image = Image.open(params.output_dir + '/' + zeros(params.start_num, output))
File "C:\Apps\Anaconda\lib\site-packages\PIL\Image.py", line 2766, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'output_dir/out_0000.png'
Using windows 10 of course.

Question: Zoom back to original Image

Thank you for taking the time to build this script its what I'm looking for and while im still brushing up on this stuff I was wondering if their was a way to modify the script to eventually end back to the main image.

So zoom in 90 frames, then keep zooming another 30 back to the original or that fact to another image that then could be zoomed into? Kinda like looping though images to images..

Cheers

readme references non-public repo "neural-zoom-pt"

unclear whether this is an oversight

Debugging folder and some improvements

Hi thanks so much for the repo, its a great start for many video art with neural nets.
Just wanted to branch off of this #4
I started a fork of the repo which mainly takes care of the PIL and tmp directory bugs.
fork

Working with the neural-style-pt
It seems I can run with adam optimizer using
python neural_zoom.py -script neural_style.py -style_image examples/inputs/coral.jpg -content_image examples/inputs/mold.jpg -model_file models/vgg19-d01eb7cb.pth -backend cudnn -optimizer adam -verbose

but not with the lbfgs, as I get this error. It seems something with the neural-style script or how it handles lbfgs?

If you think there is a place I can start digging, please let me know

Capturing style target 1
Running optimization with L-BFGS
Traceback (most recent call last):
  File "neural_style.py", line 468, in <module>
    main()
  File "neural_style.py", line 262, in main
    optimizer.step(feval)
  File "/usr/local/lib/python3.6/dist-packages/torch/optim/lbfgs.py", line 354, in step
    old_dirs.pop(0)
IndexError: pop from empty list