wansharon/deepmvs

Learning-based 3D reconstruction with SfM, MVSNet, point-cloud reconstruction, mesh reconstruction and mesh texturing

DeepMVS

Flowchart of DeepMVS

Deep learning methods have shown promising results in the area of 3D reconstruction. However, the existing 3D reconstruction projects like Colmap and OpenMVS are still based on traditional methods. Recently, the multi-view stereo methods, such as the MVSNet and its variants, have shown promising results in depth learning. Here, we build the 3D reconstruction project, which uses the learning based MVS methods for depth inferring.

The whole project is the complete 3D reconstruction system. We use the Colmap for SfM, CasMVSNet and D2HC-RMVSNet for depth inferring and OpenMVS for dense point-cloud reconstruction, mesh reconstruction and mesh texturing. We write the codes to combine them together so it can do 3D reconstruction end to end.

The online demo video is at https://www.zhihu.com/zvideo/1443954079655063552, which describes how to use the project and some 3D reconstruction results.

The highlights of our project are as follows:

1. We build the first deep learning based 3D reconstruction project, named DeepMVS.
2. DeepMVS is much faster and more accurate than OpenMVS.

Installation

Hardware

• OS: Ubuntu 16.04 or 18.04
• NVIDIA GPU with CUDA>=10.0

Software

For OpenMVS: Please refer to OpenMVS

For CasMVSNet_pl and D2HC-RMVSNet: Please refer to CasMVSNet_pl and D2HC-RMVSNet which are variants of MVSNet

Docker

We provide the docker image for environment:

docker pull minchen12345/deepmvs:latest

Usage

Note: We use the depth2dmap.py function to convert the output of MVSNets into the format of OpenMVS !!!

Run

bash demo.sh test_folder test_img_name

example:
bash demo_casmvsnet.sh example test0

Reconstruction Results

Some Results

License

Our code and dataset are released under the Apache 2.0 license.

Acknowledgement

This repository is based on Colmap, OpenMVS, CasMVSNet_pl and D2HC-RMVSNet .

TODO:

• Add the complete codes for OpenMVS
• Add SuperPoint for SfM, like https://github.com/cvg/sfm-disambiguation-colmap
• Rifine the mesh building with MVSDF