avik-pal/faststyletransfer.jl

Fast Neural Style Transfer in Julia

FastStyleTransfer

NOTE: This version works for Julia 0.6. An update to support Julia 1.0 and latest Flux, is WIP. Check the julia-1.0 branch for latest updates. It contains working code which needs to be trained. The models for the last release will not work in julia-1.0 branch. But soon new and better pre-trained models will be added.

Implementation of Fast Neural Style Transfer in Julia using Flux.jl

Usage

To install this package simply run

Pkg.clone("https://github.com/avik-pal/FastStyleTransfer.jl")

Using it is very simple. You just need to worry about 2 functions.

The first function is train. You need this in case you are training on a new style. If you ever end up training on a new style please send in a PR with the model. Now the arguments you need to pass to the function are:

1. train_data_path: Path to the MS COCO Dataset.
2. batch_size
3. η: Learning rate. Ideally set it to 0.001.
4. style_image_path
5. epochs
6. model_save_path
7. content_weight: The priority you want to assign to the content. A higher value leads to a better retainment of the original features.
8. style_weight: Same as content weight only for the style image.
9. model: Pass TransformerNet or TransformerNet2. You can pass a custom Network as well.
10. images: Total images from the COCO dataset that you want to train you model on.

The other function would be stylize. This will probably will be the only function most people shall use. So lets see its arguments:

1. image_path
2. model_path
3. save_path: Can be left blank. The image will be stored in the same directory as the image with a stylized tag.
4. display_img: If running from the terminal set it to false. Comes handy when you want to see the image without having to open the save_path.

Some speed statistics: The model runs quite fast taking only 5ms to stylize a 640 x 640 image on a P100 GPU.

Results

Future Work

1. Implement a GPU Kernel for Instance Normalization
2. Substitute Zero Padding for Reflection Padding
3. Implement the GPU Kernel for Upsampling Layer

References

1. Perceptual Losses for Real-Time Style Transfer and Super-Resolution
2. A Neural Algorithm of Artistic Style
3. Instance Normalization: The Missing Ingredient for Fast Stylization

If you are interested in the implementation look up this blog post.

faststyletransfer.jl's Issues

Precompiling failed

I tried to run your Pkg .

julia> using Pkg; Pkg.clone("https://github.com/avik-pal/FastStyleTransfer.jl")

seems OK but occurs error message the below after using FastStyleTransfer

julia> using FastStyleTransfer
[ Info: Precompiling FastStyleTransfer [ff70ec99-cc3d-52c9-809f-ecb142a82ec2]
ERROR: LoadError: ArgumentError: Package FastStyleTransfer does not have NNlib in its dependencies:
- If you have FastStyleTransfer checked out for development and have
  added NNlib as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with FastStyleTransfer
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:833
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1041
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope at none:2
 [6] eval at ./boot.jl:319 [inlined]
 [7] eval(::Expr) at ./client.jl:389
 [8] top-level scope at ./none:3
in expression starting at /home/terasaki/.julia/dev/FastStyleTransfer/src/FastStyleTransfer.jl:5
ERROR: Failed to precompile FastStyleTransfer [ff70ec99-cc3d-52c9-809f-ecb142a82ec2] to /home/terasaki/.julia/compiled/v1.0/FastStyleTransfer/I6kaR.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1187
 [4] _require(::Base.PkgId) at ./logging.jl:311
 [5] require(::Base.PkgId) at ./loading.jl:855
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:837

My PC Environment is

• OS: Ubuntu 16.04
• Julia 1.0.1
• Installed libraries

(v1.0 pkg> st
  [c52e3926] Atom v0.7.8
  [fbb218c0] BSON v0.2.1
  [336ed68f] CSV v0.4.1
  [150eb455] CoordinateTransformations v0.5.0
  [3a865a2d] CuArrays v0.8.0
  [a93c6f00] DataFrames v0.14.1
  [ff70ec99] FastStyleTransfer v0.0.0 [`~/.julia/dev/FastStyleTransfer`]
  [587475ba] Flux v0.6.7+ #master (https://github.com/FluxML/Flux.jl.git)
  [7073ff75] IJulia v1.12.0
  [6218d12a] ImageMagick v0.7.1
  [916415d5] Images v0.16.0
  [682c06a0] JSON v0.19.0
  [e5e0dc1b] Juno v0.5.3
  [dbeba491] Metalhead v0.3.0
  [438e738f] PyCall v1.18.4

Which Julia version does this repository run ?