This is a torch implementation of FCN with short-cut link and a forced same sampling mask for up/down sampling. It has been demonstrated to work well for tasks that require estimating per-pixel values. Please go to http://pbrs.cs.princeton.edu and the following paper for more information. If you use this code, please cite:
@article{zhang2016physically,
title={Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks},
author={Zhang, Yinda and Song, Shuran and Yumer, Ersin and Savva, Manolis and Lee, Joon-Young and Jin, Hailin and Funkhouser, Thomas},
journal={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2017}
}
Please contact me (yindaz AT cs DOT princeton DOT edu) if you have any problem in using the code/data/model.
You need to specify the -test_file
in config.lua
with a list of file names pointing to the images you are testing. ./image/
provides an example.
We provide several pretrained models for testing. You can download them in ./model/
.
Run
th main_test.lua -test_model ./model/sync_physic_nyufinetune.t7
and the result will be in ./result/
.
You need to specify the -train_file
in config.lua
first.
Run
th main_train_single.lua -ps ./model/train_example
The training process will start, and snapshots will be saved as ./model/train_example_iter_xxx.t7
. To finetune a model, run
th main_train_single.lua -finetune -finetune_model ./model/train_example.t7 -ps ./model/train_finetune_example
To train on synthetic image, you can find download the training data from http://pbrs.cs.princeton.edu. Specifically,
Color image
: http://pbrs.cs.princeton.edu/pbrs_release/data/mlt_v2.zip (278GB)Surface normal ground truth
: http://pbrs.cs.princeton.edu/pbrs_release/data/normal_v2.zip (27GB)Data list
: http://pbrs.cs.princeton.edu/pbrs_release/data/data_goodlist_v2.txt
To experiment on NYUv2 data,
Color image and ground truth
: http://pbrs.cs.princeton.edu/pbrs_release/nyu/nyu_data.zip. This file is converted using data from http://www.cs.nyu.edu/~deigen/dnl/ and http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.htmlTraining data list
: http://pbrs.cs.princeton.edu/pbrs_release/nyu/trainNdxs.txtTesting data list
: http://pbrs.cs.princeton.edu/pbrs_release/nyu/testNdxs.txt