hyc96/real-time-photometric-stereo-and-surface-reconstruction-on-personal-devices

Real-time-photometric-stereo-and-surface-reconstruction-on-personal-devices

Photometric stereo is a method that estimates an object’s surface depth and orientation from images of the same view taken under different light directions. Conceptually, three images with distinct light directions are sufficient in obtaining the surface normal map. However, noise is a prevalent factor in many facets of this process and industrial practice often require multiple images taken in a controlled environment. We propose to explore the robustness of shape from shading algorithms and adapt them such that real-time surface reconstruction can be achieved on personal devices. This entails using minimal input data and computation. In following sections we show that we retrieved accurate 3D surfaces of objects in real-time from a webcam live video, using only light sources from a computer screen.

Prerequisites

• cv2
• screeninfo
• mayavi
• mpl_toolkits

Usage

Run

$python main.py 

and input as prompted.

Example

• Sample object:

• Initial estimate:

• Output:

License

see the LICENSE.md file for details

References

This project is generally implemented based on:

1. Schindler, Photometric Stereo via Computer Screen Lighting for Real-time Surface Reconstruction. (2008)
2. H. Hideki, “Photometric stereo under a light-source with arbitrary motion”(1994)
3. A. Yuille, D. Snow, R. Epstein, and P. Belhumeur, “Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability”(1999)
4. V. Nozick, I. Daribo, and H. Saito, “Gpu-based photometric reconstruction from screen light” (2008)
5. V. Nozick, “Pyramidal Normal Map Integration for Real-time Photometric Stereo”(2010)

Contact me for more detailed report on the implementation.

Contact

[email protected]

real-time-photometric-stereo-and-surface-reconstruction-on-personal-devices's Issues

Suggestions to get good results

Hi,

Thanks for sharing your code.

I have been playing with photometric stereo for some time now and wanted to ask for your suggestions to get reasonable results (like shown in this repository). I have been unable to replicate those results. I tried with my face and with other objects, at various distances from the camera and screen, using a dark background and placing the camera either at the top and at the center of the screen, but with no success.

See for example the following images, each one for each lighting direction. They do not look that different:

And the result (using python main.py 2):

I was wondering if you could please comment about any step, setup or special care I am missing to get reasonable results.

Thank you!