utils/scan with colmap option

[KevinCain]

utils/scan with colmap option

[raphaelsulzer]

Hi,
thank you for your interest in the work.

For clarification, you only need the scan utility if you have ground truth meshes, which you want to sample using virtual scanning. So either you have a dataset with ground truth meshes and you want to train DGNN on it, or you want to evaluate the performance of DGNN reconstructions.

If you simply want to reconstruct a mesh starting from a point cloud, you do not need the scan utility. In this case you start with your point cloud with visibility information (e.g. in .npz file format) and extract features for DGNN like this:

utils/feat -w /pathToWorkingdir/ -i yourPointcloudFile -s npz

Then you make a custom.yaml file which points to /pathToWorkingdir/ and you can reconstruct a mesh from your point cloud.

COLMAP

feat used to be able to input a COLMAP point cloud in the way you suggest. It stopped working at some point due to some issues with Eigen versions which I could never fix.

The easiest way to get this done is to use the OpenMVS InterfaceCOLMAP tool to create an OpenMVS project from a COLMAP project. Then you can run:

utils/feat -w /pathToWorkingdir/ -i yourOpenMvsProjectFile -s omvs

If you have problems with this part, or you really want to get the COLMAP thing to work I suggest you have a look at the code in mesh-tools and open an issue there if necessary.

Hope this helps!

[KevinCain]

Thank you @raphaelsulzer for these helpful notes.

I has no trouble refreshing my OpenMVS repo and exporting pinhole cameras from a test scene, but I must not be correctly setting the scene for the dgnn helper feat.

I successfully exported an OpenMVS file from a undistorted colmap scene (images, database, the usual binary sparse reconstruction files) with:

cd ~/openMVS_build/bin
./InterfaceCOLMAP --archive-type -1 -i ~/undist/ -o Peterson.omvs

Next I invoked features in dgnn from the dgnn conda shell via:

utils/feat -w ~/undist/ -i Peterson.omvs -s omvs

This yields the following error:

-----FEATURE EXTRACTION-----

Working dir set to:
-/home/undist/
ERROR: not a valid reconstruction input

Am I ignoring other context we must supply to dgnn?

In case it's relevant, the output of my conversion from colmap to OpenMVS follows. It looks OK to me:

./InterfaceCOLMAP --archive-type -1 -i /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/ -o Peterson
21:23:22 [App     ] Build date: Feb 21 2022, 19:43:43
21:23:22 [App     ] CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 cores)
21:23:22 [App     ] RAM: 15.58GB Physical Memory 32.72GB Virtual Memory
21:23:22 [App     ] OS: Linux 5.16.10-zen1-1-zen (x86_64)
21:23:22 [App     ] Disk: 131.38GB (448.36GB) space
21:23:22 [App     ] SSE & AVX compatible CPU & OS detected
21:23:22 [App     ] Command line: InterfaceCOLMAP --archive-type -1 -i /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/ -oPeterson
21:23:22 [App     ] Reading cameras: /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/sparse/cameras.bin
21:23:22 [App     ] Reading images: /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/sparse/images.bin
21:23:22 [App     ] Reading points: /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/sparse/points3D.bin
21:23:22 [App     ] Reading patch-match configuration: /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/stereo/patch-match.cfg
21:23:22 [App     ] Reading depth-maps/normal-maps: /home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/stereo/depth_maps/ and/home/kevin/Hierarchical-Localization/outputs/sfm/sfm_superpoint+superglue/undist/stereo/normal_maps/
21:23:22 [App     ] Exported data: 60 images & 10724 vertices (26ms)
21:23:22 [App     ] MEMORYINFO: {
21:23:22 [App     ]     VmPeak:   517484 kB
21:23:22 [App     ]     VmSize:   516780 kB
21:23:22 [App     ] } ENDINFO

[raphaelsulzer]

Hi,

I'm sorry but I forgot that OpenMVS support is turned off for the statically compiled executables provided in this repository.
The reason is that I could never manage to link the OpenMVS libs statically to mesh-tools.

I'm afraid you'll either need to export your point cloud with sensor positions to an .npz or .ply file which can be read by the feat tool, or you have to compile the feat tool yourself with OpenMVS support enabled.

[KevinCain]

Thanks again @raphaelsulzer, that makes sense.

Would another option be to use your OMVS2NPZ utility, then run the provided feat binary?

[raphaelsulzer]

Yes, exactly. You can also do that!