phantom0122/3d_reconstruction_with_stereo

With Stereo Vision, 3D reconstruction based on two methods. First, correlation based algorithm which extracts disparity map then estimate depth value. The other is Feature-based algorithm which extracts features with SIFT, then use triangulation to get depth value.

3D_Reconstruction_with_Stereo

Requirement

• python 3.x
• openCV 3.x
• open3D
• Numpy

Common Step in both methods

1. Image calibration: Calibrating with checkerboard method, get intrinsic parameter of camera.
2. 3D Visualization: Visualize 3D using open3d with point cloud from depth map and RGB images.

3D Reconstuction using disparity map

Disparity map can be obtained between images, simply translated with X-axis. As depth and disparity are inversely propotional, we can estimate relative depth values then reconstruct 3D. The reconstucted 3D point clouds will be dense since depth can be inffered from disparity map, which is dense also.

Results

3D Reconstuction using Feature-based algorighm

With SIFT algorithm between two images, obtaining correspondence pairs is possible. We can find find 3D position which is the intersection of rays from each camera. The result 3D point clouds will be sparse in contrast to Disparity method since the position of points cab be calculated from feature points, which are also sparse.

Results

• Left & Right Image
  
  

• Extracted feature points
  
  

• Sparse points Clouds
  
  

Backprojection algorithm

# Loop through each pixel in the image  
for v in range(height):    
  for u in range(width):    
    # Apply equation in fig 4
    x = (u - u0) * z / fx
    y = (v - v0) * z / fy
    z = depth[v, u]