tsattler/multicamerapose

Multi Camera Pose Estimation

multicamerapose's Issues

How to use MultiCameraPose for 360 degree spherically captured scene with 3 cameras?

I came across this project after fiddling with COLMAP during last two months and still being unable to consistently get desired results on my specific use-case which relies on relative poses between cameras.

I am using COLMAP to estimate camera poses for my datasets which consist of hundreds of scenes that share some similarities. Specifically, each scene consists of spherically captured images (using a turntable) with a white background - in total either 144 images or 108 images per scene (3 or 4 vertical angles and 36 images capture horizontally per vertical angle).

Sample of images from one of the scenes:

Therefore, my camera rig consists of either 3 or 4 cameras (positioned roughly above each other) that share same intrinsics. For better illustration see screenshot from COLMAP's gui from a different scene with 3 cameras.

My current workflow is:

1. Extract features from images (colmap feature_extractor)
2. Match features across images (colmap exhaustive_matcher)
3. Take camera extrinsics from a previous scene where COLMAP was able to correctly estimate them
4. Run point triangulation using these extrinsics on the images
5. Run model refining (colmap/colmap#1057 (comment) thank you once again @tsattler!)

However, sometimes this workflow fails at step 4 and 5 when the extrinsics taken from another scene are far too different - instead of a single coherent object it creates three different reconstructions of the same object, just misaligned in height.

Extrinsics may different in:

• Height from the ground of each individual camera might be adjusted (so although their relative height difference does not change a lot it is not a constant change across scenes)

Relative extrinsics that do not change:

• Camera in each vertical angle always takes 36 photos per round, i.e difference between shots in is always 10 degrees.
• Relative "height" difference between each camera within the scene is constant, i.e. cameras are perfectly above each other and their relative pose in height does not change within the scene.

I was wondering whether there's possibility to leverage these known relative poses using proposed Multi Camera Pose Estimation and if there is then how could one go about achieving it.

I managed to get the code running and currently I am in middle of converting COLMAP's cameras.txt and images.txt to images_with_intrinsics format based on common.h file. it would be awesome if there was a sample images_with_intrinsics file.

Also I am not sure if I understand common.h correctly but how do we include points2D and points3D from COLMAP?

Any help is appreciated :)

Suggestion about the usage

Hi Prof. Torsten Sattler,
I am trying to use this project for visual localization, but I find using more images does not provide better results.
2~5 images are used whose relative poses are given by ARKit.
If I understand it correctly, the images_with_intrinsics file should provide the rotation (world to camera) and camera center of each query image, the match_file provides raw 2d-3d matches for each query. invert_Y_Z should be set to False and points_centered should be set to False.
So, do you have any idea where I might be wrong? @tsattler

How to run this code

Hello, I am sorry, it is not a issues.
I am a novice, I have finished reading your paper, and now I want to quickly run through your code to verify my understanding of your paper.Please teach me how to run your code. For example, file''images_with_intrinsics'' in the code, how to get it?
Thank you very much.

Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes Availability

Hello,

Is the paper Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes available as pdf ?

Thanks

How to compile?

Hello, I am sorry to ask how to compile this repository.

I have downloaded and compiled the dependency Poselib and modified the FindPoseLib.cmake file. However, there still exists an error about nonexistent of PoseLib.

Could you please provide some information about compilation of this project?
PS: I am not clear about the 2D-3D matches file configuration and I wonder how to get such text file. How about running Colmap or something else?

Many thanks and look forward to hearing from you.