flycanflyxd/vrn

:man: Torch7/MATLAB code for "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression"

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric Regression

Try out the code without running it! Check out our online demo here.

Please visit our project webpage for a link to the paper and an example video run on 300VW. This code is licenses under the MIT License, as described in the LICENSE file.

This is an unguided version of the Volumetric Regression Network (VRN) for 3D face reconstruction from a single image. This method approaches the problem of reconstruction as a segmentation problem, producing a 3D volume, spatially aligned with the input image. A mesh can then be obtained by taking the isosurface of this volume.

Several example images are included in the examples folder. Most of these are AFLW images taken from 3DDFA.

Software Requirements

A working installation of Torch7 is required. This can be easily installed on most platforms using torch/distro. You will also require a reasonable CUDA capable GPU. MATLAB is used to render the isosurface and a script (run.m) is provided to simplify the execution.

This project was developed under Linux. I have no idea if it will work on Windows and it is unlikely that I will be able to help you with this. If you are running Mac OS, issue #1 might be of interest to you.

Quick overview of requirements:

• Torch7 (+ nn, cunn, cudnn, image)
• MATLAB
• NVIDIA GPU, with a working CUDA installation and CuDNN.

Bulat’s face alignment code is included as a submodule. Please check his README for dependencies.

Getting Started

git clone --recursive https://github.com/AaronJackson/vrn.git
cd vrn
./download.sh

and then run “run” from MATLAB.

Using your own images

You are, of course, welcome to try out this method on your own set of images. dlib, the face detector included with Bulat’s face alignment code struggles to find side poses. You are welcome to modify the code to provide bounding boxes.

Available Options

The MATLAB “run.m” script contains a few options which you can change. Here is a very quick description of them:

• input_folder, as the name suggests, the folder to glob for JPEG images.
• output_folder, the directory to store the regressed volumes.
• model_file, the name of the Torch model to load.
• gpunum, specify which GPU to use, starting at 0.
• texture, rudimentary texture mapping by taking the 2D projections nearest neighbour.