yannnnnnnnnnnn/edgegraph3d

Multi-view Stereo 3D Edge Reconstruction

EdgeGraph3D

EdgeGraph3D is a system for multi-view stereo reconstruction of 3D edges. The system receives as input:

• a set of image observations of a scene or object
• a set of estimated camera poses and an initial cloud of 3D points, such as the one recovered by a Structure-from-Motion pipeline. EdgeGraph3D supports as input a JSON in the format of the output of OpenMVG
• a set of edge images, one per input view, in which 2D edges detected on the corresponding image are represented by white pixels over a black background. This allows the user to integrate any 2D edge detection algorithm with EdgeGraph3D

and internally generates a set of 3D edges reconstructed in the observed scene, from which the system outputs a sampling of edge-points integrated with the original SfM data. This allows the user to employ the output of EdgeGraph3D in standard reconstruction algorithms that accept as input a point cloud, to produce significantly more accurate 3D models.

Demo

Please have a look at a demo of EdgeGraph3D here

Requirements

The following libraries are needed for building EdgeGraph3D

• CMake
• OpenCV (>= 3.1)
• CGAL (>= 4.9)
• Boost
• Eigen3 (>= 3.0.5)
• GMP

The system was tested on Ubuntu 16.04 LTS with the following versions:

• CMake
• OpenCV v3.1
• CGAL v4.9
• Boost v1.61.0.2
• Eigen v3.2.5
• GMP v6.1.1

Building

On Unix platforms EdgeGraph3D can be built by running the following commands from its containing folder:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make

License

This software is published under the GNU General Public License. More information can be found here: http://www.gnu.org/licenses/

Usage

Call format:

./EdgeGraph3D [-i] images/ edge_images/ working_folder/ input_sfm_data.json output.json

    images/ : folder containing input images
    edge_images/ : folder containing input edge images w/ black background and white edges
    working_folder/ : working folder, will be used by the 3D edge reconstruction pipeline to output files
    input_sfm_data.json : input structure from motion data, in [OpenMVG](https://github.com/openMVG/openMVG) format
    output.json : output JSON file, in [OpenMVG](https://github.com/openMVG/openMVG) format, integrating the input file with edge-points sampled on edges generated by EdgeGraph3D

Optional arguments:

    -i : output debug images, i.e., 2D-to-2D polyline matching, edge-point reprojections

Utilities

A set of additional utilities is released to assist in typical computations in 3D reconstruction and related to the usage of EdgeGraph3D.

coordinate_system_transform

Transform the coordinate system in the input JSON to match the provided camera poses.

Call format:

./coordinate_system_transform input.json camera_poses.txt output.json

    input.json : input point cloud in JSON format
    camera_poses.txt : files containing the target camera poses, one per line, in the order defined by input.json, in the format "x y z"
    output.json : output transformed coordinate system point cloud in JSON format

filter

Filters outliers in a JSON.

Call format:

./filter [-s starting_point_id | -e gn_max_mse | -f force_views_filtering] input.json output.json

    input.json : input point cloud in JSON format
    output.json : output inliers point cloud in JSON format
    
Optional arguments:
    
    -s starting_point_id : filters points starting from starting_point_id based on number of observations (default is zero)
    -e gn_max_mse : maximum mean squared error to be tollerated by Gauss Newton procedure in outliers filtering
    -f force_views_filtering : forces the value for the threshold, inclusive, for observations filtering (default is set through an heuristic)

json_to_ply

Converts a JSON in a point cloud in ply format.

Call format:

./json_to_ply [-i original_images/] input.json output.ply

    input.json : input point cloud in JSON format
    output.ply : output point cloud in PLY format
    
Optional arguments:
    
    -i original_images/ : original images used to produce the provided point cloud. If this argument is provided, json_to_ply will produce a colored point cloud

Example

An example showing how to use the system has been provided in example/dtu006.

Cite

If you use EdgeGraph3D or its algorithms for publication, please cite the paper:

@inproceedings{bignoli2018edgegraph3d,
  title={Multi-View Stereo 3D Edge Reconstruction},
  author={Bignoli, Andrea and Romanoni, Andrea and Matteucci, Matteo},
  booktitle={Applications of Computer Vision (WACV), 2018 IEEE Winter Conference on},
  year={2018},
  organization={IEEE}
}

Currently, the paper is accessible at: https://arxiv.org/abs/1801.05606

Acknowledgements

EdgeGraph3D has been designed and developed by Andrea Bignoli at Politecnico di Milano (Italy), with the supervision and support of PhD Andrea Romanoni and Prof. Matteo Matteucci. We recommend the reconstruction algorithm provided by Andrea Romanoni, https://github.com/andresax/Manifold-Reconstructor, to produce 3D models from the enhanced point clouds generated through EdgeGraph3D.

Contact

Andrea Bignoli mail: andrea.bignoli [at] gmail.com