How to extract 2D-3D array from colmap database to feed "pose_refinement" function?

[colmap]

How to extract 2D-3D array from colmap database to feed "pose_refinement" function?

[mihaidusmanu]

You will need to extract the matches from the database and parse the 3D model using the following utilities from the COLMAP repository:

• https://github.com/colmap/colmap/blob/dev/scripts/python/read_write_model.py
• https://github.com/colmap/colmap/blob/dev/scripts/python/database.py

Another useful link is https://colmap.github.io/format.html which explains the COLMAP reconstruction output format.

For each query image, you go through its matching pairs and only keep matches to keypoints that appear in the 3D model - in this way you infer a 2D-3D match from the 2D-2D ones in the COLMAP database.

Some version of this is already implemented in hloc - I think https://github.com/cvg/Hierarchical-Localization/blob/master/hloc/localize_sfm.py#L99.

[vahidEtt]

Hi @mihaidusmanu. Thanks for the quick reply. I will look at them.

So you mean I should run the whole reconstruction and after it, I should extract 2d-3d points? At which stage in colmap reconstruction, we have a 3D model that I can extract 2d-3d points? I thought after "colmap matches_importer" I can extract them.

[mihaidusmanu]

colmap matches_importer simply imports 2D-2D matches. There is no mapping involved up until that point. The mapper or point_triangulator (when poses are known) produce the sparse 3D map based on local feature matches. You can take a look at the CLI tutorial for the different COLMAP mapping steps - see https://colmap.github.io/cli.html.

[vahidEtt]

Ok. Thanks!