caor-mines-paristech/colibri-vr-unity-package

This is the Unity package for COLIBRI VR, the Core Open Lab on Image-Based Rendering Innovation for Virtual Reality.

colibri-vr-unity-package's Issues

Ghosting artifacts in ULR for virtual scenes

Hi,
I have a question regarding the implementation of the unstructured lumigraph rendering: Why can I see ghosting artifacts for virtual scenes created with the provided acquisition interface using the high quality (non-optimized) ULR shader? I would expect that the UL rendering is near perfect because everything I provided (images, depth, cameras and mesh) is ground truth. There should not be any inaccuracies that could explain these artifacts.
I also checked the original ULR paper but could not find any hints for this problem (except for the proxies maybe?). Do you have any idea why this is happening?

Very interesting system! Some small findings and a question

Hi,

Thanks for the great work and making this available! I've been looking into lightfield technology and how it can be used to capture and play back 3D scenes (in VR, AR, regular 3D) using readily available technology, and this looks like a great toolbox for that! And it's even integrated into Unity already, wow.

Here are a few small things I had to do to get it up and running on my computer, and at the end a question on the usage:

1. In PerViewMeshesQSTR.compute I replaced the first dot in the cginc file names with a double dot, otherwise the file couldn't be found.

    #include "../../CGIncludes/CoreCG.cginc"
    #include "../../CGIncludes/CameraCG.cginc"
    #include "../../CGIncludes/ColorCG.cginc"

2. When I ran a sparse reconstruction the first time, it completed very quickly and then said it was succesful, but actually there were no results in the sparse folder. I ran the command line from the Console in a command prompt and found that Colmap was missing glew32.dll. So I ended up copying the files in the Colmap 'lib' folder into the 'bin' folder and that solved it :-)

3. After computing a reconstruction of the 'door' dataset, I was a bit surprised to see this message: COLMAP camera type SIMPLE_RADIAL is not currently supported by COLIBRI VR.. Suggestion to put the currently unsupported camera models between brackets in the Colmap Editor panel (or something like that)

4. I ran the reconstruction again, with the undistorted images and PINHOLE camera model, and got a good result I think. Just a question: when I viewed it using the Rendering script, I could not see a result larger than in the attached image , by varying the Focal length and Max blend angle parameters. The list of Blending methods is also smaller than in the tutorial video, maybe because the downloadable unitypackage is a bit older than the development version?

Looking forward to doing additional experiments with it (e.g. to export to a VR experience) !

Lex

Per-view depth map format and build question

Hi,

This is a really nice work! I tried to use my own global mesh/images for ULR rendeirng and it works fine. Now I want to supply depth map to get the per-view mesh, I am having trouble finding the format for the correct depth map format. They are saved as 8-bit png files when I ran the acquisition package. Do you have any documents stating what the format it should be?

The other issue is when I tried to build a final binary, i don't think the data are copied over and when i launch it just shows a empty scene. Is this expected?

Thanks!

Issue with the blending method type: Disk-blended focal surfaces

Hi, I'm having an issue with the blending method type: Disk-blended focal surfaces. It simply does not do anything. I've gone through your tutorial several times thinking maybe I missed something. But still nothing. Not entirely sure why It's not working, is there any solution to this problem. Thanks

Import Existing Colmap Model

Hello thanks for the cool project!

Just wanted to know if there is a way to import an external generated colmap model?

I see nothing when I press play button in the unity editor and when I stop play, the editor crashes. I've bundled the data.

COLMAP Masks

Hello,

Very impressive work, thank you for sharing.
I am testing it with a multi-camera setup. With our previous experiments I found that
masking helps a lot with COLMAP reconstruction speed and geometry.
Is there a way to specify the mask images and place them somewhere?
Thanks in advance

ULR shader compatibility with metal

Awesome system that I’m really enjoying playing with. I am trying to use the global mesh ULR method but seem to be having an issue with the ULR shader on my Mac. The shader shows up pink in the Unity editor.

Parse and Save button stayed disabled

Hi,

I had to make two minor changes in StanfordLightfieldHelperEditor.cs

//            GUI.enabled = isGUIEnabled && _targetObject.colorCount > 0;
// LvdS: it seems the variable above is filled in ParseCameraSetup(), but this has not been executed at this point
            GUI.enabled = isGUIEnabled && _targetObject.dataHandler.sourceColorCount > 0;

            EditorGUILayout.Space();
            label = "Parse and save";
            tooltip = "Camera setup will be parsed and saved here: \"" + _targetObject.dataHandler.dataDirectory + "\".";

//            if(_targetObject.colorCount < 1)
// LvdS: same as above
            if(_targetObject.dataHandler.sourceColorCount < 1)
                tooltip = "No color images were found in the \"images\" folder of the directory: \"" + _targetObject.dataHandler.dataDirectory + "\".";

Are these changes correct?

And a small question: at the end of the Amethyst tutorial video you move something around which causes the highlight on the object to shift, but what operation is that? Panning the Scene view camera? Or translating a certain object?