adepierre/caffe_began

Caffe/C++ implementation of Boundary Equilibrium Generative Adversarial Networks paper for face generation

Caffe_BEGAN: A Caffe/C++ implementation of Boundary Equilibrium Generative Adversarial Networks algorithm for face generation

Implementation of BEGAN algorithm for face generation. Here are some examples (128x128 images) with a model trained on 26 epochs of CelebA dataset (gamma = 0.5). The holes are due to the high gamma value. A smaller value of gamma (as 0.3 for example) would result in better images but with less diversity.

The evolution of the training from epoch 0 to epoch 25:

Dependencies

This code relies on Caffe main branch with only two slight modifications in solver.hpp:

• ApplyUpdate function must be moved from protected to public in Solver class definition
• iter_ must be moved from protected to public in Solver class definition

OpenCV and boost are also used, but you should already have them installed if you have Caffe.

As Caffe doesn't have a L1-Loss layer, I have used existing layers to build the equivalent (the four last layers of launch_files/discriminator.prototxt).

I have only tested this code on Windows with Visual Studio 2015, but it should also be able to run on Linux and OS X provided that Caffe is correctly installed with the above modification.

Building

The project uses CMake as a building tool. Once you have correctly built and compiled the code, you should be abble to launch the program for both training and testing.

Testing

To test a model, set the testing parameters and then launch launch_files/test.bat. A file with the weights of a trained model is provided if you just want to see it generating faces: launch_files/BEGAN_Generator_iter_325000.caffemodel. It will display some images generated from random inputs.

Training

First, you need to create a folder with all the training images. You can have as many images as you want (jpg, png or bmp): they are loaded and unloaded on a separate thread during training so you do not need tens of Gb of RAM. If you want to use CelebA dataset, please use the aligned and cropped images because the program will always use only the center 128x128 pixels (or the whole image if it is smaller) and resize it to the desired size.

Then you can train a model: set the training parameters as you want and then launch launch_files/train.bat. Images are saved (5 per epoch) to show the evolution of the learning process. Loss values are also saved in a csv file to monitor the training. Prototxt files (generator.prototxt and discriminator.prototxt) are regenerated everytime the program is launched (because they have to change with batch size, image size, input dimension ...).

During training, several files are created:

• caffemodel and solverstate to save networks states
• k_t files to save k_t values along with caffe snapshots
• log file to save losses
• preview_values.csv to keep the random inputs used to display images during training

All those files are used if you want to stop the training and continue it later (it took me almost 3 days to train the above 128x128 model on a Nvidia GTX 1070).

If everything is correct, your learning curves should look like those ones:

Resuming Training

To resume training, you just need to add the snapshosts parameters to the train.bat file and launch it again. If the program finds the log file and the preview_values file, they should be loaded as well. New ones are created otherwise (so you won't have consistent faces between the images before and after you stop).

License

MIT

caffe_began's Issues

error at compilation : trying to access to a protected variable

Hello,

thank you for that library.
I am trying to compile it : everything is fine with the "cmake .". When I push make I get :

Scanning dependencies of target Caffe_BEGAN
[ 33%] Building CXX object CMakeFiles/Caffe_BEGAN.dir/src/NN_Agent.cpp.o
/path/to/Caffe_BEGAN/src/NN_Agent.cpp: In member function ‘void NN_Agent::Train(const std::vector<std::vector >&, const std::vector<std::vector >&)’:
/path/to/Caffe_BEGAN/src/NN_Agent.cpp:129:32: error: ‘void caffe::Solver::ApplyUpdate() [with Dtype = float]’ is protected within this context
solver_generator->ApplyUpdate();
^
In file included from /usr/include/caffe/caffe.hpp:15:0,
from /path/to/Caffe_BEGAN/include/NN_Agent.h:6,
from /path/to/Caffe_BEGAN/src/NN_Agent.cpp:1:
/usr/include/caffe/solver.hpp:99:16: note: declared protected here
virtual void ApplyUpdate() = 0;
^~~~~~~~~~~
/path/to/Caffe_BEGAN/src/NN_Agent.cpp:130:20: error: ‘int caffe::Solver::iter_’ is protected within this context
solver_generator->iter_++;
^~~~~
In file included from /usr/include/caffe/caffe.hpp:15:0,
from /path/to/Caffe_BEGAN/include/NN_Agent.h:6,
from /path/to/Caffe_BEGAN/src/NN_Agent.cpp:1:
/usr/include/caffe/solver.hpp:113:7: note: declared protected here
int iter_;
^~~~~
/path/to/Caffe_BEGAN/src/NN_Agent.cpp:148:36: error: ‘void caffe::Solver::ApplyUpdate() [with Dtype = float]’ is protected within this context
solver_discriminator->ApplyUpdate();
^
In file included from /usr/include/caffe/caffe.hpp:15:0,
from /path/to/Caffe_BEGAN/include/NN_Agent.h:6,
from /path/to/Caffe_BEGAN/src/NN_Agent.cpp:1:
/usr/include/caffe/solver.hpp:99:16: note: declared protected here
virtual void ApplyUpdate() = 0;
^~~~~~~~~~~
/path/to/Caffe_BEGAN/src/NN_Agent.cpp:149:24: error: ‘int caffe::Solver::iter_’ is protected within this context
solver_discriminator->iter_++;
^~~~~
In file included from /usr/include/caffe/caffe.hpp:15:0,
from /path/to/Caffe_BEGAN/include/NN_Agent.h:6,
from /path/to/Caffe_BEGAN/src/NN_Agent.cpp:1:
/usr/include/caffe/solver.hpp:113:7: note: declared protected here
int iter_;
^~~~~
CMakeFiles/Caffe_BEGAN.dir/build.make:62: recipe for target 'CMakeFiles/Caffe_BEGAN.dir/src/NN_Agent.cpp.o' failed
make[2]: *** [CMakeFiles/Caffe_BEGAN.dir/src/NN_Agent.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/Caffe_BEGAN.dir/all' failed
make[1]: *** [CMakeFiles/Caffe_BEGAN.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Do you have any idea of what happen ?.

Thank you for your time !

Unable to train the model

Hi,

So I have successfully built the Caffe_BEGAN binary under Ubuntu, and I made some modifications to the test.bat and made it into test.sh. It runs perfectly and generated nice faces.

But when I am trying to do the same modifications to the train.bat, I encountered segmentation fault. Would you please advice me how to write a bash script which can make your program run the training process?

I have attached both my test.sh and train.sh as txt files, test runs but train gives me segmentation fault. Please help!

When I do ./train.sh, I got the following:

./train.sh: line 21: 61279 Segmentation fault ./Caffe_BEGAN --train=1 --weights_gen= --weights_discr= --z_dim=128 --h_dim=128 --hidden_dim=64 --image_size=128 --solver_gen=../launch_files/solver_generator.prototxt --solver_discr=../launch_files/solver_discriminator.prototxt --batch_size=16 --snapshot_generator= --snapshot_discriminator= --snapshot_k_t= --preview_generator=preview_values.csv --number_batch_loaded=125 --start_epoch=0 --end_epoch=25 --training_dataset=../Data

My Data folder contains 202 thousand CeleA .jpg images.

test.txt
train.txt

Resuming training does not work

Hi,

I set the parameters as follows:

#!/bin/bash

./Caffe_BEGAN ^
--train=1 ^
--weights_gen= ^
--weights_discr= ^
--z_dim=128 ^
--h_dim=128 ^
--hidden_dim=64 ^
--image_size=128 ^
--solver_gen=solver_generator.prototxt ^
--solver_discr=solver_discriminator.prototxt ^
--batch_size=16 ^
--snapshot_generator=BEGAN_Generator_iter_500000.solverstate ^
--snapshot_discriminator=BEGAN_Discriminator_iter_500000.solverstate ^
--snapshot_k_t=k_t_iter_500000.txt ^
--preview_generator=preview_values.csv ^
--number_batch_loaded=125 ^
--start_epoch=0 ^
--end_epoch=25 ^
--training_dataset=../Data/CelebA

I gave *.sloverstate files and corresponding t_t.

The program always generates an image before the training, so I checked the Generation_epoch_0.bmp, it's pure noise. I thought the first image generated should be faces since I have resumed training from iteration 500000.

Did I set the parameters wrong?

Thank you,
Sebastian

Unable to Run Caffe_BEGAN

Hi There!

I have done the following:

In the ROOT Directory I have ran "cmake ./". I get the following:

-- Configuring done
-- Generating done
-- Build files have been written to: /p/prometheus/private/govind2/Caffe_BEGAN.

When I run the test exec, I get the following error:

./launch_files/BEGAN_Generator_iter_325000.caffemodel: line 6: syntax error near unexpected token `)'
generator_ip1�/BEGAN_Generator_iter_325000.caffemodel: line 6: InnerProduct�

And it prints some random string.

I tried running the train.bat file, I get the following error:
../bin/Release/Caffe_BEGAN.exe No such file or directory

There is nothing in the bin directory. Am I missing something ?

Regards
Govind

Windows Builds

Hello,
could you add compiled windows binaries to the project ?
It's farily hard to build the project, for me.