Generalisze_Absolute_Pose usage

[colmap]

Generalisze_Absolute_Pose usage

[mihaidusmanu]

I do not really understand why you want to use the generalized absolute pose estimation. In general, this is used for localizing a rig with known calibration (intrinsics and relative poses between the cameras of the rig) inside of a 3D world map. In your case, you already have these world transforms (from ARKit) and you only have a single iPhone / iPad camera not a rig (unless you treat the whole trajectory as a rigid rig, but that will have problems with drift).

If you, for instance, want to register your iPhone trajectory with respect to another 3D map, that is where the 3D points for generalized absolute pose estimation will come from - you will have to establish correspondences between the images in the current trajectory and the images in the reference model and the lift the keypoints from the reference model to 3D using e.g. a sparse SfM model or a lidar scan / mesh.

[theponpon]

Hi Mihal,

Thanks for the quick answer.
I should have explained what I am trying to achieve:

• I have a recorded AR session composed of a set of synchronised images, extrinsics and intrinsics
• I can extract visual features from images and pair features across images

My purpose is to use the bundle adjustment features of Colmap to refine the ARKit extrinsics that can sometime drift, especially in the scenario of closing loops.
I want to use the extrinsics from ARKit as initialisation for the bundle adjustment.
I would like to avoid having to deal with the colmap dataset

How would you recommend to use pycolmap to achieve this?

Thanks in advance,

Pierre

[mihaidusmanu]

PyCOLMAP currently has very limited functionality - we only have bindings for localization purposes: single image pose estimation, rig pose estimation, and relative pose (E / F). There are people working on integrating mapping as well (#20), but it might take a while until it is available.

However, for your task, you can simply run from the command line colmap point_triangulator to triangulate a 3D model from the ARKit poses and intrinsics followed by colmap bundle_adjuster to refine the poses and intrinsics . Please refer to https://colmap.github.io/faq.html#reconstruct-sparse-dense-model-from-known-camera-poses for more details regarding the point triangulator.

[torkos]

I wish you answered the original question, since I'm also confused regarding the inputs of rig_absolute_pose_estimation.
There's a list of lists of 3d points. And a list of lists of 2d points. If I had 2 cameras, left and right would the first 2d list be left keypoints, and 3d would be the corresponding original 3d, and second list the right 2d points with corresponding 3d in the original?

Or do all the keypoints have to be related somehow (left and right match up)?

[mihaidusmanu]

The parameters are lists of lists of 2D/3D points, each inner list corresponding to a camera. There is no consistency needed between cameras. In your case points2D = [points2D_left, points2D_right] and points3D = [corr_points3D_left, corr_points3D_right].

You can check out the implementation for more details https://github.com/colmap/pycolmap/blob/master/estimators/generalized_absolute_pose.cc#L298-L310

[torkos]

This is perfect, thank you.