jakobwilm/slstudio

SLStudio -- Real Time Structured Light. A comprehensive software suite for capturing and reconstructing 3D scenes with a camera-projector pair.

SLStudio -- Real Time Structured Light

This software is designed to enable anyone to implement a custom 3D structured light scanner using a single camera and light projector. It is modular and has a focus on processing speed, enabling real-time structured light capture at 20 Hz and more. When using standard commercial projector and a webcam, the obtainable speed is lower due to the lack of hardware triggering.

When using the software in academic work, please consider citing the following publication.

Wilm et al., SLStudio: Open-Source Framework for Real-Time Structured Light, IPTA 2014 IEEE Xplore

Usage is permitted under GPLv3 license since August 2021. Hence, any distributed (internally or externally) derivative work must be published openly. Use before that date constitutes copyright violation. A commercial license can be obtained for commercial users not wanting to disclose their source code (see License section). Feature/functionalty development on a contract basis are also offered. Please contact the author if interested.

Realtime setup

Demo videos

Compiling and installing

SLStudio is being developed with qmake (QtCreator). The /src/SLStudio.pro file is the project file which contains all information about the project and its dependencies needed for compiling. It is has a number of dependencies that you need to install before being able to compile the program on your machine:

• Qt 5.X (4.x or 6.x could be used with relatively few modifications)
• OpenCV 4.x or newer (3.x can be used with relatively few modifications)
• Point Cloud Library 1.7 or newer
• VTK 7 (newer versions do not seem to support PCLVisualizer)
• Boost
• Eigen
• FLANN
• GLEW
• Depending on your camera: libdc1394, FlyCapture API, XIMEA xiApi, IDS Imaging uEye API

The project has successfully been compiled on Ubuntu 22.04, OS X 10.9 and Windows 7. The tested and recommended OS is Ubuntu 22.04.

Using components

The mathematical/algorithmic and hardware interface parts reside in individual classes from the presentation layer and business logic. The algorithmic classes (Codec*, Triangulator, Camera, Projector) have very few external dependencies and are designed to be used on their own or in different environments. The Matlab interface shows what this might look like.

While the algorithm classes could easily be separated into a static/dynamic library, we currently see little value in doing so, and recommend to copy the relevant classes to a new project. Please note license terms.

Ubuntu 22.04

Ubuntu also has all of the dependencies available as packages (except camera libraries). Running the following line should have you (almost) set:

    sudo apt-get install libpcl-dev libopencv-dev libvtk9-qt-dev libglew-dev freeglut3-dev qtcreator libusb-1.0-0-dev qmake6 libxrandr-dev

One advantage of using Ubuntu is that you are able to render structured light patterns on a secondary X screen, which does not interfere with your main screen in which Gnome desktop and the SLStudio GUI run. Usually, this is an unusual use-case, as normally you are able to move the mouse or windows onto the second screen or use ALT-Tab. However, by setting up two X Screens in xorg.conf with a gap in between them, you can make Gnome completely ignore the second screen so it is only SLStudio that draws onto it. This also depends on your graphics driver supporting multiple X screens (work with current proprietary nVidia and AMD drivers).

OS X 10.X

On OS X, the dependencies are available through MacPorts. You will not get completely independent screens, so your measurements may be corrupted by GUI activity. Otherwise the program runs well.

Windows 7/8/10

On Windows machines, OpenCV and PCL are available as downloads, but since everything needs to be ABI compatible, it is probably best to compile the dependencies from source with the same compiler (e.g. MSVC) which is rather time-consuming but possible. It is some time since we have last compiled on Windows, but all of the code should be compatible.

Camera APIs

Because high performance operation of cameras often requires vendor- or device-specific features or setup, we choose not to use an intermediate layer such as OpenCV to communicate with the camera. Currently, wrappers for a number of camera APIs is provided, and we invite anyone who implements others to contribute them. Be aware that we have only tested on a limited number of cameras, and some of the configurations performed may only apply to our specific models (e.g. images are assumed to be in grayscale). In any case, you may need to modify some of the camera code to get this to work with hardware triggering (required for real time performance).

Matlab wrappers

The project also contains Matlab mex-wrappers for the OpenGL projector and Camera. This makes it possible to e.g. determine the gamma response of your camera-projector setup, or other debugging tasks. The mex-wrappers are compiled from Matlab by running matlab/make.m. Tested on Ubuntu only.

Running

Projector source

SLStudio can run in different projector modes. For projection you can choose OpenGL projection, which renders the structured light patterns on "screen 2". You would then use an HDMI connection to the projector. Specifically for LightCrafter (LightCrafter 3500), LightCrafter4500 and newer, the software can communicate over USB with these projectors, which can preload the patterns.

Camera Triggering

In the preference pane, software and hardware triggering can be selected. The calibration procedure allways uses software triggering. For real-time structured light, you need hardware triggering from the projector to the camera by means of a trigger signal cable. Few projectors provide the trigger output signal, but with some effort you may be able to source one from the HDMI vsync pin. LightCrafter4500 has trigger output pins and is the recommended device. You will have to produce a trigger cable with the specific headers of your projector and camera. With commercial projectors you can usually project 60 patterns per second at most (LC4500 allows for 120 8-bit patterns per second). Usually the camera requires a pause between two consecutive exposures, and you will need to waste one refresh period. With LC4500 we can project 60 grayscale patterns per second resulting in up to 20 point clouds per second.

Some cameras have only a single I/O port, e.g. Ximea MQ013RG-E2. This is enough for frame triggering, but synchronization with regard to the sequence is difficult to achieve. For this problem, a triggering circuit bases on an Arduino microcontroller was developed. The source code found in trigger_circuit can be uploaded to an Arduino and takes two triggering pulses from the projector (frame start and sequence start) and generates one output signal (long pulse for sequence start and short pulses for all other frames). Camera frames are marked with the I/O status shortly after triggering which allows us to very robustly determine the sequence start.

Calibration

Calibration of your hardware setup determines the internal camera and projector parameters, and the relative position of the projector relative to the camera. The calibration procedure only is seen in the video above, and requires a flat board with a grey-white checkerboard printed onto it. Good calibration requires a large number of calibration positions covering all of the sensor field of view and with some foreshortening (angle of the calibration board relative to view-axis). You can use e.g. https://calib.io/pages/camera-calibration-pattern-generator to generate a PDF for the checkerboard.

Codec

SLStudio serves as a structured light platform. A number of encoding strategies has been implemented, and these differ greatly in the number of patterns, accuracy and robustness to e.g. shiny or semitransparent surfaces. For many applications, 2x3 phase shifting provides good results. It uses 6 patterns, so you will reach 10Hz point cloud update frequency with the appropriate hardware.

Recommended settings

Please note that some parts of this software are still experimental, while others have been matured. For reliable point cloud capture we recommend the following choices:

• Ubuntu 20.04
• Newer dedicated nVidia graphics card
• Ximea or Point Grey camera
• Calibration in 20 positions
• Calibration board with 10 x 10 saddle points
• Software triggering
• 2x3 Phase Shifting

For real-time (10Hz +) performance we recommend the following:

• LightCrafter4500 configured for 120Hz HDMI gray-scale projection (e.g. configure using /tools/lc4500startup)
• Ximea MQ013RG-E2
• Hardware triggering

Support

While we are interested in providing our software to a rich audience and make real-time structured light available to many user groups, it usually requires a fair amount of customization and knowledge to build these systems. With the current state of this project and the challenges in hardware, you do need C++ programming experience to make it work with your specific setup, unless it consists of the exact components listed above. We can answer some questions pertaining to software and hardware, but cannot provide low-level support. We are though very interested in incorporating relevant and meaningful improvements and bug fixes, so if you have such, please contact us or make a pull request!

License

SLStudio did not provide an explicit license before August 2021. Any usage of the software before that date is technically a copyright violation. Since August 2021, the software is released under a dual license scheme. Users may use it under GPLv3 (see LICENSE.md), or obtain a commercial license, which allows for distribution of derivative works without disclosing any source code. Contact [email protected] if you are interested in obtaining a commercial license.

slstudio's Issues

Does it generate a 3D Mesh?

Hi Jakob, does it generate a 3D Mesh on the fly? What's the output file of this after triggering the different lightproejctions? I've seen you used the LC4500. Can you suggest any new model which supports HD, in case this delivers a better output? thx

Different Outputs

Good Day Sir,

I was trying to work on your repository recently for my University Project. When I run the application, and scan the example data, the saved .pcd and .ply outputs result different than the "pointCloud.ply" outputs presented in the example_data/... folders. I wonder if there is an extra process needed to come up with presented "pointCloud.ply" after the scan is done.

Ps: I get meaningful .pcd and .ply outputs from skull_3ps and skull_gray; however, the outputs are not the same as "example_data/skull_3ps/pointCloud.ply" and "example_data/skull_gray/pointCloud.ply" respectively. Resulting output files has more noise than the example ones. Resulting outputs are below.

skull_3ps Result:

example_data/skull_3ps/pointCloud.ply:

skull_gray Result:

example_data/skull_gray/pointCloud.ply:

Sincerely.

Strange calibration and reconstruction results

Hi,

I always got incorrect calibration result. The camera calibration is good but the projector calibration result is totally incorrect, as well as the stereo calibration. The reprojection error of projector or stereo is more than e+02. The point cloud result is just a plane without depth information.
I have tried to load the correct calibration file, but the 3D result is messed up. I also tried the matlab code and made my gamma response linear.
Do you know how to resolve this problem?

Bests,
Yujia

when the slstudio was compiled, errors occurred as follows:

when the slstudio was compiled, errors occurred as follows:

error: undefined reference to `pcl::tracking::KLDAdaptiveParticleFilterTracker<pcl::PointXYZRGB, pcl::tracking::ParticleXYZRPY>::insertIntoBins(std::vector<int, std::allocator >, std::vector<std::vector<int, std::allocator >, std::allocator<std::vector<int, std::allocator > > >&)'
could you please give some suggestions?

how to handle the dynamic scene reconstruction

 i am a postgratude student from ningbo university china, i have used this project to generate point cloud in static scene,  but my project is how to use structure light to reconstruct a motion entity.  i do not know whether this project  is also suitablefor dynamic scene or not  and theoretically motion compensation should be considered but i did not observe the related code for motion compensation in this project.  simultaneously  i notice the video for this project seems to work well  for generating point clouds of a  moving head?  so i am confused.   could you please give me some suggestion on this issue and   wonder if there are some materials or code  available for structured light reconstruction in dynamic scene,  looking forward to get your reply, thank you very much.

Gtk-ERROR **: GTK+ 2.x symbols detected.

The programs builds fine with few modifications, but the executable fails to run with this message

(SLStudio:6008): Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported
Trace/breakpoint trap (core dumped)

How to I force it to use a GTK-2 ?

Fail to run matlab determineGammaResponse script - Exception error?

Hi Jakob,

I apologise for raising two issues in one day, but having looked at the code in the matlab directory, we do not understand what is going wrong seeing how early it bails out.

We have built the matlab mex files successfully after the fix, but now when running determineGammaResponse we get an error on line 5 where it creates an projector object. Here is the output on the error below. Also we have been able to use the camera and projector with the SLStudio calibration.

DLP Lightcrafter
XIMEA XiQ MG013MG-E2
Ubuntu 16.04 64-bit
matlab R2018a

Warning: The following error was caught while executing 'Projector' class destructor:
Error using ProjectorMex
Input must be a real uint64 scalar.

Error in Projector/delete (line 24)
            ProjectorMex('delete', this.objectHandle);

Error in Projector (line 18)
        function this = Projector(screenNum)

Error in determineGammaResponse (line 5)
p = Projector(1); 
> In Projector (line 18)
  In determineGammaResponse (line 5) 
Unexpected unknown exception from MEX file..

Error in Projector (line 19)
            this.objectHandle = ProjectorMex('new', screenNum);

Error in determineGammaResponse (line 5)
p = Projector(1);

Matlab code - Error when running mex files.

Hi Jakob,

Using the updated version of the matlab code, we have seemingly successfully built the executables (console output below). Then when running the .mexa64 files in matlab are getting the errors:

>> ProjectorMex
Error using ProjectorMex
First input should be a command string less than 64 characters long.

and

>> CameraMex
Error using CameraMex
First input should be a command string less than 64 characters long.

This seems as if arguments are being expected? To run each of these .mexa64 files we are calling ProjectorMex or CameraMex in the console without any parentheses. Are we doing this incorrectly?

Gcc version: 6.3.0
Ubuntu 16.04 64-bit
matlab R2018a

Build output:

>> make
Verbose mode is on.
... Looking for compiler 'g++' ...
... Executing command 'which g++' ...Yes ('/usr/bin/g++').
... Executing command 'g++ -print-file-name=libstdc++.so' ...Yes ('/usr/lib/gcc/x86_64-linux-gnu/6/libstdc++.so').
... Executing command 'g++ -dumpversion' ...Yes ('6.3.0').
... Executing command 'which g++' ...Yes ('/usr/bin/g++').
... Looking for folder '/usr' ...Yes.
... Executing command 'g++ -dumpmachine' ...Yes ('x86_64-linux-gnu').
Found installed compiler 'g++'.
Set INCLUDE = /usr/lib/gcc/x86_64-linux-gnu/6.3.0/include;/usr/include/c++/6.3.0;/usr/include/c++/6.3.0/x86_64-linux-gnu;/usr/include/c++/6.3.0/backward;/usr/lib/gcc/x86_64-linux-gnu/6.3.0/include;/usr/include/c++/6.3.0;/usr/include/c++/6.3.0/x86_64-linux-gnu;/usr/include/c++/6.3.0/backward;
Options file details


	Compiler location: /usr/bin/g++
	Options file: /usr/local/MATLAB/R2018a/bin/glnxa64/mexopts/g++_glnxa64.xml
	CMDLINE2 : /usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_975137339462_1885/CameraMex.o /tmp/mex_975137339462_1885/Camera.o /tmp/mex_975137339462_1885/CameraIIDC.o /tmp/mex_975137339462_1885/CameraXIMEA.o /tmp/mex_975137339462_1885/cpp_mexapi_version.o   -ldc1394  -lm3api   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o CameraMex.mexa64
	CXX : /usr/bin/g++
	DEFINES : -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE 
	MATLABMEX : -DMATLAB_MEX_FILE 
	CFLAGS : -fexceptions -fPIC -fno-omit-frame-pointer -pthread
	CXXFLAGS : -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11
	INCLUDE : -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include"
	CXXOPTIMFLAGS : -O -DNDEBUG
	CXXDEBUGFLAGS : -g
	LDXX : /usr/bin/g++
	LDFLAGS : -pthread -Wl,--no-undefined 
	LDTYPE : -shared
	LINKEXPORT : -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/mexFunction.map"
	LINKEXPORTVER : -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map"
	LINKLIBS : -ldc1394  -lm3api   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++
	LDOPTIMFLAGS : -O
	LDDEBUGFLAGS : -g
	MWCPPLIB : "/usr/local/MATLAB/R2018a/sys/os/glnxa64/libstdc++.so.6"
	OBJEXT : .o
	LDEXT : .mexa64
	SETENV : CC="gcc"
                CXX="/usr/bin/g++"
                CFLAGS="-fexceptions -fPIC -fno-omit-frame-pointer -pthread -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE "
                CXXFLAGS="-fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE "
                COPTIMFLAGS="-O -DNDEBUG"
                CXXOPTIMFLAGS="-O -DNDEBUG"
                CDEBUGFLAGS="-g"
                CXXDEBUGFLAGS="-g"
                LD="gcc"
                LDXX="/usr/bin/g++"
                LDFLAGS="-pthread -Wl,--no-undefined  -shared -ldc1394  -lm3api   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/mexFunction.map""
                LDDEBUGFLAGS="-g"
	GCC : /usr/bin/g++
	CPPLIB_DIR : /usr/lib/gcc/x86_64-linux-gnu/6/libstdc++.so
	VER : 6.3.0
	GCCDIR : /usr
	GCC_TARGET : x86_64-linux-gnu
	MATLABROOT : /usr/local/MATLAB/R2018a
	ARCH : glnxa64
	SRC : "/home/physics/project/slstudio/matlab/CameraMex.cpp";"/home/physics/project/slstudio/matlab/../src/camera/Camera.cpp";"/home/physics/project/slstudio/matlab/../src/camera/CameraIIDC.cpp";"/home/physics/project/slstudio/matlab/../src/camera/CameraXIMEA.cpp";"/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp"
	OBJ : /tmp/mex_975137339462_1885/CameraMex.o;/tmp/mex_975137339462_1885/Camera.o;/tmp/mex_975137339462_1885/CameraIIDC.o;/tmp/mex_975137339462_1885/CameraXIMEA.o;/tmp/mex_975137339462_1885/cpp_mexapi_version.o
	OBJS : /tmp/mex_975137339462_1885/CameraMex.o /tmp/mex_975137339462_1885/Camera.o /tmp/mex_975137339462_1885/CameraIIDC.o /tmp/mex_975137339462_1885/CameraXIMEA.o /tmp/mex_975137339462_1885/cpp_mexapi_version.o 
	SRCROOT : /home/physics/project/slstudio/matlab/CameraMex
	DEF : /tmp/mex_975137339462_1885/CameraMex.def
	EXP : "CameraMex.exp"
	LIB : "CameraMex.lib"
	EXE : CameraMex.mexa64
	ILK : "CameraMex.ilk"
	MANIFEST : "CameraMex.mexa64.manifest"
	TEMPNAME : CameraMex
	EXEDIR : 
	EXENAME : CameraMex
	OPTIM : -O -DNDEBUG
	LINKOPTIM : -O
	CMDLINE1_0 : /usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/CameraMex.cpp" -o /tmp/mex_975137339462_1885/CameraMex.o
	CMDLINE1_1 : /usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/Camera.cpp" -o /tmp/mex_975137339462_1885/Camera.o
	CMDLINE1_2 : /usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/CameraIIDC.cpp" -o /tmp/mex_975137339462_1885/CameraIIDC.o
	CMDLINE1_3 : /usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/CameraXIMEA.cpp" -o /tmp/mex_975137339462_1885/CameraXIMEA.o
	CMDLINE1_4 : /usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_975137339462_1885/cpp_mexapi_version.o



Building with 'g++'.
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/CameraMex.cpp" -o /tmp/mex_975137339462_1885/CameraMex.o
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/Camera.cpp" -o /tmp/mex_975137339462_1885/Camera.o
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/CameraIIDC.cpp" -o /tmp/mex_975137339462_1885/CameraIIDC.o
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/camera/CameraXIMEA.cpp" -o /tmp/mex_975137339462_1885/CameraXIMEA.o
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_975137339462_1885/cpp_mexapi_version.o
/usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_975137339462_1885/CameraMex.o /tmp/mex_975137339462_1885/Camera.o /tmp/mex_975137339462_1885/CameraIIDC.o /tmp/mex_975137339462_1885/CameraXIMEA.o /tmp/mex_975137339462_1885/cpp_mexapi_version.o   -ldc1394  -lm3api   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o CameraMex.mexa64
Recompile embedded version with '-DMATLAB_MEXCMD_RELEASE=R2017b'
/usr/bin/g++ -c -DWITH_CAMERAIIDC  -DWITH_CAMERAXIMEA  -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/camera" -I"/usr/local/lib" -I"/opt/XIMEA/include"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_975137339462_1885/cpp_mexapi_version.o -DMATLAB_MEXCMD_RELEASE=R2017b
/usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_975137339462_1885/CameraMex.o /tmp/mex_975137339462_1885/Camera.o /tmp/mex_975137339462_1885/CameraIIDC.o /tmp/mex_975137339462_1885/CameraXIMEA.o /tmp/mex_975137339462_1885/cpp_mexapi_version.o   -ldc1394  -lm3api   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o CameraMex.mexa64
MEX completed successfully.
Verbose mode is on.
... Looking for compiler 'g++' ...
... Executing command 'which g++' ...Yes ('/usr/bin/g++').
... Executing command 'g++ -print-file-name=libstdc++.so' ...Yes ('/usr/lib/gcc/x86_64-linux-gnu/6/libstdc++.so').
... Executing command 'g++ -dumpversion' ...Yes ('6.3.0').
... Executing command 'which g++' ...Yes ('/usr/bin/g++').
... Looking for folder '/usr' ...Yes.
... Executing command 'g++ -dumpmachine' ...Yes ('x86_64-linux-gnu').
Found installed compiler 'g++'.
Set INCLUDE = /usr/lib/gcc/x86_64-linux-gnu/6.3.0/include;/usr/include/c++/6.3.0;/usr/include/c++/6.3.0/x86_64-linux-gnu;/usr/include/c++/6.3.0/backward;/usr/lib/gcc/x86_64-linux-gnu/6.3.0/include;/usr/include/c++/6.3.0;/usr/include/c++/6.3.0/x86_64-linux-gnu;/usr/include/c++/6.3.0/backward;
Options file details



	Compiler location: /usr/bin/g++
	Options file: /usr/local/MATLAB/R2018a/bin/glnxa64/mexopts/g++_glnxa64.xml
	CMDLINE2 : /usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_982710491912_1885/ProjectorMex.o /tmp/mex_982710491912_1885/ProjectorOpenGL.o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o /tmp/mex_982710491912_1885/cpp_mexapi_version.o   -lGL  -lGLU  -lX11  -lXxf86vm  -lGLEW   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o ProjectorMex.mexa64
	CXX : /usr/bin/g++
	DEFINES : -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE 
	MATLABMEX : -DMATLAB_MEX_FILE 
	CFLAGS : -fexceptions -fPIC -fno-omit-frame-pointer -pthread
	CXXFLAGS : -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11
	INCLUDE : -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include"
	CXXOPTIMFLAGS : -O -DNDEBUG
	CXXDEBUGFLAGS : -g
	LDXX : /usr/bin/g++
	LDFLAGS : -pthread -Wl,--no-undefined 
	LDTYPE : -shared
	LINKEXPORT : -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/mexFunction.map"
	LINKEXPORTVER : -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map"
	LINKLIBS : -lGL  -lGLU  -lX11  -lXxf86vm  -lGLEW   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++
	LDOPTIMFLAGS : -O
	LDDEBUGFLAGS : -g
	MWCPPLIB : "/usr/local/MATLAB/R2018a/sys/os/glnxa64/libstdc++.so.6"
	OBJEXT : .o
	LDEXT : .mexa64
	SETENV : CC="gcc"
                CXX="/usr/bin/g++"
                CFLAGS="-fexceptions -fPIC -fno-omit-frame-pointer -pthread -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE "
                CXXFLAGS="-fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE "
                COPTIMFLAGS="-O -DNDEBUG"
                CXXOPTIMFLAGS="-O -DNDEBUG"
                CDEBUGFLAGS="-g"
                CXXDEBUGFLAGS="-g"
                LD="gcc"
                LDXX="/usr/bin/g++"
                LDFLAGS="-pthread -Wl,--no-undefined  -shared -lGL  -lGLU  -lX11  -lXxf86vm  -lGLEW   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/mexFunction.map""
                LDDEBUGFLAGS="-g"
	GCC : /usr/bin/g++
	CPPLIB_DIR : /usr/lib/gcc/x86_64-linux-gnu/6/libstdc++.so
	VER : 6.3.0
	GCCDIR : /usr
	GCC_TARGET : x86_64-linux-gnu
	MATLABROOT : /usr/local/MATLAB/R2018a
	ARCH : glnxa64
	SRC : "/home/physics/project/slstudio/matlab/ProjectorMex.cpp";"/home/physics/project/slstudio/matlab/../src/projector/ProjectorOpenGL.cpp";"/home/physics/project/slstudio/matlab/../src/projector/OpenGLContext.Unix.cpp";"/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp"
	OBJ : /tmp/mex_982710491912_1885/ProjectorMex.o;/tmp/mex_982710491912_1885/ProjectorOpenGL.o;/tmp/mex_982710491912_1885/OpenGLContext.Unix.o;/tmp/mex_982710491912_1885/cpp_mexapi_version.o
	OBJS : /tmp/mex_982710491912_1885/ProjectorMex.o /tmp/mex_982710491912_1885/ProjectorOpenGL.o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o /tmp/mex_982710491912_1885/cpp_mexapi_version.o 
	SRCROOT : /home/physics/project/slstudio/matlab/ProjectorMex
	DEF : /tmp/mex_982710491912_1885/ProjectorMex.def
	EXP : "ProjectorMex.exp"
	LIB : "ProjectorMex.lib"
	EXE : ProjectorMex.mexa64
	ILK : "ProjectorMex.ilk"
	MANIFEST : "ProjectorMex.mexa64.manifest"
	TEMPNAME : ProjectorMex
	EXEDIR : 
	EXENAME : ProjectorMex
	OPTIM : -O -DNDEBUG
	LINKOPTIM : -O
	CMDLINE1_0 : /usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/ProjectorMex.cpp" -o /tmp/mex_982710491912_1885/ProjectorMex.o
	CMDLINE1_1 : /usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/projector/ProjectorOpenGL.cpp" -o /tmp/mex_982710491912_1885/ProjectorOpenGL.o
	CMDLINE1_2 : /usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/projector/OpenGLContext.Unix.cpp" -o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o
	CMDLINE1_3 : /usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_982710491912_1885/cpp_mexapi_version.o


Building with 'g++'.
/usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/ProjectorMex.cpp" -o /tmp/mex_982710491912_1885/ProjectorMex.o
/usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/projector/ProjectorOpenGL.cpp" -o /tmp/mex_982710491912_1885/ProjectorOpenGL.o
/usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/home/physics/project/slstudio/matlab/../src/projector/OpenGLContext.Unix.cpp" -o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o
/usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_982710491912_1885/cpp_mexapi_version.o
/usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_982710491912_1885/ProjectorMex.o /tmp/mex_982710491912_1885/ProjectorOpenGL.o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o /tmp/mex_982710491912_1885/cpp_mexapi_version.o   -lGL  -lGLU  -lX11  -lXxf86vm  -lGLEW   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o ProjectorMex.mexa64
Recompile embedded version with '-DMATLAB_MEXCMD_RELEASE=R2017b'
/usr/bin/g++ -c -DMATLAB_DEFAULT_RELEASE=R2017b  -DUSE_MEX_CMD   -D_GNU_SOURCE -DMATLAB_MEX_FILE  -I"../src/projector"  -I"/usr/local/MATLAB/R2018a/extern/include" -I"/usr/local/MATLAB/R2018a/simulink/include" -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c++11 -O -DNDEBUG "/usr/local/MATLAB/R2018a/extern/version/cpp_mexapi_version.cpp" -o /tmp/mex_982710491912_1885/cpp_mexapi_version.o -DMATLAB_MEXCMD_RELEASE=R2017b
/usr/bin/g++ -pthread -Wl,--no-undefined  -shared -O -Wl,--version-script,"/usr/local/MATLAB/R2018a/extern/lib/glnxa64/c_exportsmexfileversion.map" /tmp/mex_982710491912_1885/ProjectorMex.o /tmp/mex_982710491912_1885/ProjectorOpenGL.o /tmp/mex_982710491912_1885/OpenGLContext.Unix.o /tmp/mex_982710491912_1885/cpp_mexapi_version.o   -lGL  -lGLU  -lX11  -lXxf86vm  -lGLEW   -Wl,--as-needed -Wl,-rpath-link,/usr/local/MATLAB/R2018a/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/bin/glnxa64" -Wl,-rpath-link,/usr/local/MATLAB/R2018a/extern/bin/glnxa64 -L"/usr/local/MATLAB/R2018a/extern/bin/glnxa64" -lMatlabDataArray -lmx -lmex -lmat -lm -lstdc++ -o ProjectorMex.mexa64
MEX completed successfully.
>> 

Problem with 3D reconstruction

Hi Jakob,

Upon calibration, we receive the following results. Then using this calibration and the attached preferences, we get a 3D reconstruction that looks like the other image attached.

It appears to us that the object shown in the 3D reconstruction should be at the crossing of the axis lines, but is instead displaced some amount rendering it hard to view.

We are also under the impression that the feint 3D reconstruction shown is not, in fact, a 3D reconstruction, but instead just our camera frames, which seems bizarre.

Do you have any ideas or resolutions on how we can fix this?

We are using ubuntu 16.04 64-bit with the latest branch of slstudio, with a minor change to use pcl 1.7 instead of 1.8 (changing the version in the slstudio.pro file). Is this version change the issue?

Hardware:
DLP lightcrafter 3500 (with a modification to output a 5V trigger signal by moving some resistors on its PCB)
Ximea Xiq MQ013MG-E2

`========================================
Camera Calibration:

• cam_error:
  0.8049
• Kc:
  [19950.707, 0, 639.5;
  0, 19950.707, 271.5;
  0, 0, 1]
• kc:
  [17.54, 0, 0, 0, 0]
  Projector Calibration:
• proj_error:
  96.3
• Kp:
  [9287.7764, 0, -2014.4105;
  0, 9287.7764, 678.02643;
  0, 0, 1]
• kp:
  [-1.55, 0, 0, 0, 0]
  Stereo Calibration:
• stereo_error:
  75.11
• Rp:
  [0.958996, 0.056349218, 0.27776149;
  0.068554565, 0.9048152, -0.42024967;
  -0.27500355, 0.42205957, 0.86385113]
• Tp:
  [-479.1, 1337, -1775]
  Seq error 1 cam:0.6059 proj:30.7
  Seq error 2 cam:0.6556 proj:72.49
  Seq error 3 cam:0.4669 proj:68.86
  Seq error 4 cam:0.5649 proj:44.43
  Seq error 5 cam:0.3978 proj:72.67
  Seq error 6 cam:0.9967 proj:75.99
  Seq error 7 cam:0.4299 proj:60.53
  Seq error 8 cam:0.553 proj:34.98
  Seq error 9 cam:0.6184 proj:65.53
  Seq error 10 cam:0.4295 proj:76.21
  Seq error 11 cam:0.4578 proj:66.2
  Seq error 12 cam:0.5199 proj:44.22
  Seq error 13 cam:0.6177 proj:74.11
  Seq error 14 cam:0.686 proj:82.97
  Seq error 15 cam:0.4333 proj:97.14
  Seq error 16 cam:0.7851 proj:68.74
  Seq error 17 cam:0.5311 proj:95.8
  Seq error 18 cam:0.5311 proj:39.25
  Seq error 19 cam:0.3853 proj:55
  Seq error 20 cam:1.188 proj:151.4
  Seq error 21 cam:0.7961 proj:95.36
  Seq error 22 cam:0.4313 proj:62.24
  Seq error 23 cam:0.5 proj:54.2
  Seq error 24 cam:0.5939 proj:88.12
  Seq error 25 cam:0.5404 proj:60.88
  `
  

Some questions about the decode algorithm named DecoderPhaseShift2x3!!

Hello, thank you for your work about the SLStudio. I have a question about the DecoderPhaseShift2x3 class in the SLStudio. Thank you for your answer. This decode algorithm uses single frequency phase shifting images (3 images) and phase cue patterns(3 images) to get the unwrapped absolute phase image using. In my work, I used three different frequency phase shifting patterns at least to get the absolute phase image. So, could you tell me the theory about DecoderPhaseShift2x3 or just give me the paper name you refered?
thank you very much. @jakobwilm @snesgaard @ereh

Compilation error in windows

I am using Windows 7 with Qt 5.6.0. When I build this project, there is no bug, but 442 warnings and the compilation failed. The Compile Output includes many this kind of errors:
[ cl : Command line warning D9024 : unrecognized source file type 'Project/slstudio-master/build-SLStudio-Desktop_Qt_5_4_2_MSVC2012_OpenGL_32bit-Debug"', object file assumed
cl : Command line warning D9027 : source file 'Project/slstudio-master/build-SLStudio-Desktop_Qt_5_4_2_MSVC2012_OpenGL_32bit-Debug"' ignored]

The final error that exit the process:
[jom: C:\Users\zhang.2669\Documents\Qt Project\slstudio-master\build-SLStudio-Desktop_Qt_5_4_2_MSVC2012_OpenGL_32bit-Debug\Makefile.Debug [debug\moc_SLStudio.cpp] Error 1
jom: C:\Users\zhang.2669\Documents\Qt Project\slstudio-master\build-SLStudio-Desktop_Qt_5_4_2_MSVC2012_OpenGL_32bit-Debug\Makefile [debug] Error 2
16:24:35: The process "C:\Qt\Tools\QtCreator\bin\jom.exe" exited with code 2.
Error while building/deploying project SLStudio (kit: Desktop Qt 5.4.2 MSVC2012 OpenGL 32bit)
When executing step "Make"
16:24:35: Elapsed time: 02:39.]

Do you know how to resolve this problem? Thank you!

Gray code algorithm

Hi,

Hopefully some very quick questions.

Has the gray code algorithm been tested and does it work? (I am getting some strange results)

For a projector with a resolution of 854 x 480 what are the main values that need to be changed in the code to make this work. (Nhorz Nvert etc...)

Thanks

"Perform Calibration" quits, & no dropdown access to webcam in preferences menu

Built on gt5 using SLStudio.pro configuration.
I can run SLStudio from inside my newly created build folder, but when I click "perform calibration" SLStudio quits with a "The program has unexpectedly finished." Is this because I have no calibration.xml file?

I can use cheese to preview my webcam (so I know I can see it), but the dropdown menu in Preference inside SLStudio only show "SLStudio Virtual Camera" as an option. Doesnt recognize my USB camera.

Despite two issues above, I tried running without calibration and with "SLStudio Virtual Camera" (Preferences: single point cloud acquisition, 2x3 pattern phase shift, Screen 0 projector, software trigger) and my point clouds are empty, presumably because I am not using my usb webcam.

What matlab shows when I open .pcd output file:

# .PCD v.7 - Point Cloud Data file format
VERSION .7
FIELDS x y z rgb
SIZE 4 4 4 4
TYPE F F F I
COUNT 1 1 1 1
WIDTH 640
HEIGHT 512
POINTS 327680
DATA ascii
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255
0.000000 0.000000 0.000000 255

Any tips on calibration and webcam usage would be greatly appreciated!
(Also pardon simplicity in questions; first endeavor into c++)

Morris

xubuntu running 16.04.
Playstation eye camera
Sony MP-CL1A projector

Calibration error

Hello Mrs.
Why does the error of the projector always reach tens or hundreds?

About Triangulator.cpp

Hi, I'm not very clear with the algorithm of the triangulation. Would you please cite the relevant paper?

Ubuntu dependencies

Hi, I'm just compiling your project and had to install two additional packages, so the dependencies for Ubuntu (16.04) become:

sudo apt-get install qt5-qmake libvtk6-qt-dev libpcl-dev libopencv-dev libglew-dev qtcreator qt5-default libusb-1.0-0-dev

qt5-default is required on systems that have both Qt4 and Qt5 installed. Without this package, qmake would fail on my machine.

libusb-1.0-0-dev is a dependency in src/SLStudio.pro.

Frequncy enquirment

Hi ！
When I ran the software with my own data, I was unable to complete the final reconstruction.I suspect the frequency is wrong.
Does the software have requirements for the frequency of projected fringe?Is it necessary to project fringe patterns at specific frequencies?
Thank you very much.

A flash back problem

Hi, thank you very much for sharing such a great project, which is a great help to all.
I have recently encountered a bug. When I try to load camera parameters, the software reports an error and flashes. I have tried for a long time to fix this problem, so I would like to ask you for help.
Thank you very much.
OS : Ubuntu 22.04
Compilation：QTCreator

Strange calibration results

Hi Jakob,
I've tried to calibrate it with a commercial projector and Basler USB camera, calibration went ok, but after that "2+1 Patter Phase shift" option gives me really wavy (instead of flat) surface. Do you have any ideas why it could happened?

Thanks.

Windows binary

Can you provide a Windows binary? I am not good at compiling

LC4500 Pattern Mode with libusb

Hi @jakobwilm, can I ask you for tips about what patterns (sequence) should I load for LC4500 to be able to run your software in pattern mode? I know the procedure to make patterns with DLP Control Software, but I believe your software utilize particular pattern sequence with particular timeouts (exposure and period).
May be you can share the screenshots DLP Control Software with sequence settings or you have a f/w for LC4500 with this patterns you can upload to your this project?
Thank you in advance.

My calibration is wrong

Through the structured-plot I calculate the point of the projector, But the point is wrong .because I use the chessboard to calibrate , so the recovery point in projector should be like a chessboard , but the result is not in order.

Hardware Trigger

I've a sync cable between camera and LightCrafter 4500 and I switched SLStudio to Hardware Triggering, but from projector output (TRIG_OUT1) I see no pulse at the scope to trigger the camera. In a project README file you also mention that for realtime setup we have to use /tools/lc4500startup to prepare projector, but I did not find LCR_SetTrigOutConfig function which makes output setup. Am I missed something? Thank you.

PS: Another question - why grey-white checkerboard matters? Why not black-white?

120Hz

@jakobwilm @ereh

In readme file you saying that "LightCrafter4500 configured for 120Hz HDMI gray-scale projection (e.g. configure using /tools/lc4500startup)"but indeed lc4500startup does not switch to 120Hz LC4500. Do you have any recommendation how to set 120Hz as output for second screen (Ubuntu)?

Thank you.

Calibration problem

Hi,

I am going to perform the 3d scanning of the object using the slstudio in educational purposes.
I use the Imaging Source camera DMK22BUC03 (mono),Panasonic LCD Projector PT-L520E, and computer with processor 4x AMD A8-6600K APU with Radeon(tm) HD Graphics and operating system - Ubuntu 15.04 (x86_64).
Following the provided readme file I have successfully compiled the sources, having previously installed all the dependencies.
So, I have Qt version 4.8.6, QMake 2.01a, OpenCV 2.4.9, PCL 1.7 and VTK 5.8.
In order to make it work with my camera I wrote specified camera class containing the methods, which override the virtual ones of Camera.h class and finally can see a good mono picture in calibration window.
I have connected the projector to system unit using the VGA cable, monitor to system unit using a DVI one.
And try to perform calibration of camera and projector using the provided software and having the following settings:
Preferences->Pattern Mode - 2x3 Pattern Phase Shift, Camera - Imaging Source DMK22BUC03, Trigger Mode - Software.
Everything I have in projector checkbox is 1) Screen 0: 1280x1024 , 2) SLStudio Virtual Screen 3) LC3000 API and 4)LC4500 API.
However if I check the #1 I'll see only the white screen after pressing the "Perform Calibration" button (if I don't use projector, and 2 white screens if I use it and 2 monitors DVI and VGA with help of ARandR). If I check #2 I'll see the stream from my camera but no patterns appear after "snap"
(as they do in your video https://www.youtube.com/watch?v=swszXuPxGZI ).

Can you recommend something?

Looking forward to your answer.
Thank you in advance.

License?

Hi,

Thanks for sharing an interesting project! I am interested in using parts of this project in a closed source application. Is that possible? What is the license of slstudio?

Regards,
Nils

Sample calibration file and images to test 2 x 3 phase

Hi Jakob,

Would you have a sample calib.xml and associated 2 x 3 phase set of images I could test with.

I am having trouble balancing getting a good calibration with my setup. I can see stripes in the calibration images so I know it's not right. Whenever I change the video driver settings to get better values for the camera & calibration errors by removing the stripes it compromises the phase patterns produced when actually scanning an object. Having a reference sample calib and images might help me figure out what is going on

Thanks

the theory of the reconstruction?

strip or sepckle?
is there any papers or tutorial about the encoding of the speckle translation?

beginner guide

Hello,

I'm interested to use your software but I'm quite a beginner in almost all the techno you are using.
Tried to follow the install instruction on ubuntu and get stuck in Qtcreator, when I load your project and compiling it it says a package is missing
pcl_visualisation-1.8
is there any page which try to explain with more details how slstudio works ?
like the classic basic workflow to render a 3D image ? (open Qt creator -> open Slstudio project -> compile program)
Thanks

2×3 means 2 frewuencies and 3 phase shift?

2×3 means 2 frewuencies and 3 phase shifts?

No user forum - Pls Respond if you have worked with MicroPhaseShift etc

Hi,

I feel a bit silly trying to contact the community via an Git Issue - so maybe my issue is that there is not user forum for this project ;-)

Anyway, if anyone who sees this has SLStudio working and tried MicroPhaseShift, or indeed anything other than 2x3 and Grey code, could you please let me know how well it worked.

The reason I ask is that the readme file recommends 2x3 explicitly - even through more advanced algorithms such as Micro Phase Shifting are present. Why recommend 2x3 with all the inherent phase unwrapping issues over better choices? Are the implementations of Micro Phase Shifting etc somehow inferior?

build with modern VTK

hello,

this is not a bug but just a assistance request.
I've tried to build the project ,
for that , I've cloned VTK (v8) but seems you guys are using another version because libs are absolutely different from what you are referring to in your PRO file.
have you tried to build with latest VTK?

subpixel interpolation - greycode

Hi,

There is a comment in the GrayCode decoder codec as follows

// Binarize frames. TODO: subpixel interpolation.

How would you go about implementing sub pixel interpolation for the gray code patterns when acquired through the camera? Not looking for code but more the approach on how to tackle this?

Could the same technique work for the three phase images in 2x3 codec?

Thanks

Dot

Other Projector

Hi ~
Does this software directly control other projectors? In addition to the LightCrafter4500, LightCrafter3500 that the author mentioned.
Thank you very much.

Multi scan merge

Hi,

i was wondering if there is anything in there that will merge scans if we need to do multiple scans to complete a full 360 scan?

Regards,

Per

Wave in point cloud

Hi,

I managed to get the software running on windows but I am seeing a wave in the 3D point cloud.

I have attached a picture below. Any idea what this is?

The point cloud itself if of a flat piece of paper and was scanned using the 2 x 3 patterns in one direction (vertical patterns). When I look at your point cloud that in in your video of the face it is very clean. What is going wrong do you think with my point cloud. The problem is very repetitive.

I am using the lightcrafter 4500 in the setup and the calibrations is less than 1 for both the camera and projector.

Thanks in advance

Dot

Build Code Error

Hi,
I've tried to build the project ,but got the error code about that:

slstudio-master\src\tracker\CorrEstOrgProjFast.h:33: error: C4716: 'CorrEstOrgProjFastpcl::PointXYZRGBNormal,pcl::PointXYZRGBNormal,float::clone' : must return a value

and your source code is:
boost::shared_ptr< CorrespondenceEstimationBase<PointSource, PointTarget, Scalar> > clone() const{}

Can you help solve this error?

opencv camera

but how can I set the opencv camera to higher resolution I only work at 640*480 I have tried a thousand ways but as if it does not read the parameters

Calibration of projector

Hi,

Thanks for sharing this code.

I am a little confused on how the projector is calibrated so I was hoping someone could explain the theory. Does the projector not need to project it's own pattern in order to calibrate it?

In particular I am confused about how the projector corner coordinates are calculated using the code snippet below. I understand the camera calibration part.

for(unsigned int j=0; j<qci.size(); j++){

            const cv::Point2f &qcij = qci[j];

            // Collect neighbor points
            const unsigned int WINDOW_SIZE = 10;
            std::vector<cv::Point2f> N_qcij, N_qpij;

            // avoid going out of bounds
            unsigned int starth = max(int(qcij.y+0.5)-WINDOW_SIZE, 0u);
            unsigned int stoph  = min(int(qcij.y+0.5)+WINDOW_SIZE, frameHeight-1);
            unsigned int startw = max(int(qcij.x+0.5)-WINDOW_SIZE, 0u);
            unsigned int stopw  = min(int(qcij.x+0.5)+WINDOW_SIZE, frameWidth-1);

            for(unsigned int h=starth; h<=stoph; h++){
                for(unsigned int w=startw; w<=stopw; w++){
                    // stay within mask
                    if(mask[i].at<bool>(h,w)){
                        N_qcij.push_back(cv::Point2f(w, h));

                        float upijwh = up[i].at<float>(h,w);
                        float vpijwh = vp[i].at<float>(h,w);
                        N_qpij.push_back(cv::Point2f(upijwh, vpijwh));
                    }
                }
            }
            //std::cout << i << " findHomography " << N_qcij.size() << " " << N_qpij.size() << std::endl;
            // if enough valid points to build homography
            if(N_qpij.size() >= 50){

// std::cout << i << " findHomography" << std::endl;
// translate qcij into qpij using local homography
cv::Mat H = cv::findHomography(N_qcij, N_qpij, cv::LMEDS);
if(!H.empty()){
cv::Point3d Q = cv::Point3d(cv::Mat(H*cv::Mat(cv::Point3d(qcij.x, qcij.y, 1.0))));
cv::Point2f qpij = cv::Point2f(Q.x/Q.z, Q.y/Q.z);

                    qpi_a.push_back(qpij);
                    qci_a.push_back(qci[j]);
                    Qi_a.push_back(Qi[j]);
                }
            }
        }

Also is there a reason why one white image of the calibrations grid is not taken as opposed to 12 using the phase patterns. Is this way more accurate?

Thanks

Triangulator algorithm and some questions

Hi man, I have studied on SLStudio for some time and now have a few questions to ask you。Could you have time to help me solve these questions?
1、What is the position of the phase algorithm in Triangulaotr? It is neither the traditional fixed position of the phase algorithm, nor is the flexible position of the phase algorithm, I did not understand the algorithm in Triangulator.cpp
2、I have tested the first six codec algorithms, but the point cloud model shown in the SLPointCloudWidget is still incorrect and can clearly see the shadows left by the black and white stripes on the surface of the object, causing the depth of the object's surface as a sine change, I think this is caused by the solution is not correct.

Hope to get your reply as soon as possible，thanks! @

State of Triangulator::triangulateFromUpVp

Hi,

I've been tinkering around with SLstudio and I noticed the warning that the method used for triangulation when horizontal and vertical are activated is incomplete. Could you indicate the state of the method, e.g. is it somewhat working, what exactly is missing and may be provide a link to something describing the algorithm you wanted to implement?

Thanks for sharing your awesome work!
Cheers,
jan0e

Guide on how to compile?

Without being a lvl 9000+ Nerd on linux, how do you even compile and run it?
or on windows?

X Error of failed request: GLXBadDrawable

When I compile with " SOURCES += projector/OpenGLContext.GLFW.cpp", the screen 1 shows, but whenever I scan or calibrae camera and projector, the following errors shows like :
X Error of failed request: GLXBadDrawable
Major opcode of failed request: 154 (GLX)
Minor opcode of failed request: 16 (X_GLXVendorPrivate)
Serial number of failed request: 184
Current serial number in output stream: 186
Any advice could help me? I've been looking for so long.
Thanks!

About calibrate

Thanks for your works.
I test calibate using the example data, I got:
Camera Calibration:

• cam_error:
  0.3244
• Kc:
  [2204.1396, 0, 843.5;
  0, 2204.1396, 675.5;
  0, 0, 1]
• kc:
  [-0.05676, 0, 0, 0, 0]
  Projector Calibration:
• proj_error:
  2.576
• Kp:
  [9329.2891, 0, 874.30811;
  0, 9329.2891, 1442.9962;
  0, 0, 1]
• kp:
  [3.604, 0, 0, 0, 0]
  Stereo Calibration:
• stereo_error:
  20.48
• Rp:
  [0.88852656, -0.11886059, 0.44316214;
  0.13676967, 0.99056607, -0.0085391086;
  -0.43796644, 0.068198368, 0.89640081]
• Tp:
  [-429.7, -208.6, 1961]
  maybe something wrong with the Projector Calibration and Stereo Calibration.
  my setting is following:
  chechersize = 15
  checkerrows = 9;
  checkcols = 13;

my edit code:

`std::vectorcv::Mat readImages(std::string path)
{
std::vectorcv::Mat images;
std::vectorstd::string names;
cv::glob(path, names);
for(auto name: names)
{
std::cout << name << std::endl;
cv::Mat mat = cv::imread(name, cv::IMREAD_UNCHANGED);
images.push_back(mat);
}
return images;
}

void SLCalibrationDialog::on_calibrateButton_clicked() {

// Disable interface elements
ui->calibrateButton->setEnabled(false);
ui->listWidget->setEnabled(false);

// Stop live view
killTimer(liveViewTimer);
reviewMode = true;
ui->snapButton->setText("Live View");

calibrator->reset();

// Note which frame sequences are used
activeFrameSeqs.clear();

QListWidget* widget = ui->listWidget;
for (int i = 0; i < widget->count(); i++) {
QListWidgetItem* item = widget->item(i);
if (item->checkState() == Qt::Checked) {
// vectorcv::Mat frameSeq(frameSeqs[i].begin(),
// frameSeqs[i].begin() +
// calibrator->getNPatterns());
std::string path = rootPath + "/" + item->text().toStdString();
std::cout << "add path:\n" << path << std::endl;
std::vectorcv::Mat frameSeq = readImages(path);
calibrator->addFrameSequence(frameSeq);
frameSeqs.push_back(frameSeq);
activeFrameSeqs.push_back(i);
}
}
calibrator->checkerSize = ui->checkerSizeBox->value();
calibrator->checkerCols = ui->checkerColsBox->value();
calibrator->checkerRows = ui->checkerRowsBox->value();

// Perform calibration
calib = calibrator->calibrate();

// Re-enable interface elements
ui->calibrateButton->setEnabled(true);
ui->listWidget->setEnabled(true);
ui->saveButton->setEnabled(true);
}

void SLCalibrationDialog::on_listWidget_itemSelectionChanged() {
std::cout << "on_listWidget_itemSelectionChanged" << std::endl;
// If selection was cleared
if (ui->listWidget->selectedItems().isEmpty())
{
std::cout << "empty" << std::endl;
return;
}

// Stop live view
killTimer(liveViewTimer);
reviewMode = true;
ui->snapButton->setText("Live View");

int currentRow = ui->listWidget->currentRow();
//ui->videoWidget->showFrameCV(frameSeqs[currentRow].back());
std::string path;
path = rootPath + "/" + ui->listWidget->currentItem()->text().toStdString();
std::cout << "current path: " << path << std::endl;
std::vectorstd::string names;
cv::glob(path, names);
if(names.size() > 0)
{
cv::Mat mat = cv::imread(names[0]);
if(!mat.empty())
ui->videoWidget->showFrameCV(mat);
}
}`

thank you in advance.

boost version?

Compiled Windows version

Hi

Can you provide a Windows binary of some sort? I am not good at anything programming or compiling, so i can't really use this.

CorrEstOrgProjFast

boost::shared_ptr< CorrespondenceEstimationBase<PointSource, PointTarget, Scalar> > clone() const{}
says must return somthing...
can you correct this.

Error running latest branch

Using ubuntu 16.04 64-bit and the latest branch of slstudio, I am encountering these two errors when running the calibration. The error occurs shortly after opening the window, without using any of the buttons or options in the calibration window.

Using earlier branches this error did not occur. The second error here seems to be something in the code, rather than our hardware, however I am unsure on the openCV error.

Error messages:

...
CameraXIMEA: Error at xiGetImage (10)
OpenCV Error: Assertion failed (s >= 0) in setSize, file /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/core/src/matrix.cpp, line 116
terminate called after throwing an instance of 'cv::Exception'
  what():  /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/core/src/matrix.cpp:116: error: (-215) s >= 0 in function setSize

...
CameraXIMEA: Error at xiGetImage (10)
OpenCV Error: Insufficient memory (Failed to allocate 23258632277584 bytes) in OutOfMemoryError, file /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/core/src/alloc.cpp, line 52
terminate called after throwing an instance of 'cv::Exception'
  what():  /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 23258632277584 bytes in function OutOfMemoryError

Hardware:
DLP lightcrafter 3500
Ximea Xiq MQ013MG-E2

Projector question

Hi Jacob,
First of all thank you for all your hard work, I'm trying to follow your steps, and I have question about Projector setup.
I have:

• TI DLPLCR4500EVM (same as you have) connected via USB (can see it via lsusb)
• Ubuntu 16.04.1 LTS (Release: 16.04)
• Basler USB3 camera (will make pull request as soon as it will start work all together)

but when I run ProjectorTest from ~/slstudio/slstudio-master/src/projector it's shows
`Screen 0: width 2624, height 900
terminate called after throwing an instance of 'char const*'
also wondering should I upgrade the projector firmware first(now v.2.0.0) and may be preset-up it? Because right now it shows patterns in red via USB (and in B/W via HDMI), but I want to trigger camera, so I need USB mode.

few questions

1- can i use another DLP3010EVM-LC ? its about 103 Hz fresh rate for Real Time Structure
2- is there an option to use 2 cameras ? without code modification in triangulation
( i will use Ximea MQ013RG-E2 )
3- does gpu processing works on windows ?