Comments (9)
You will need a way to lift the 2D points from the image with pose to 3D - depth or a sparse 3D model for instance.
In this repository, we suppose you already have access to 2D<->3D correspondences for absolute pose estimation so building the sparse model is out of scope.
from pycolmap.
In this case, points2D would be the 2D keypoints in a different image corresponding to the 3D points lifted in the current image. I am unsure what you are trying to achieve. Maybe if you explain that, I could be of more help. What is your current evaluation scenario? Do you want to get the relative pose between two images or do you want to get the absolute pose of an image given a pointcloud?
from pycolmap.
Do you have descriptors for both the point-cloud and the local features in the image? If so, then all you need to implement yourself is matching between the descriptors. Next you will have to give the 2D points and their 3D point correspondences to pycolmap.absolute_pose_estimation
as shown in the README and that will return you the pose.
from pycolmap.
@mihaidusmanu I have not managed to retrieve the 3D points from the 2D points using the pose information of the image. Could you please suggest me a method that I can use to retrieve the 3D points from a 2D image with a known pose?
from pycolmap.
Given an image with camera pose (world-to-image) P and intrinsics matrix K, you can back-project a pixel location (x, y) to 3D by computing P^{-1} @ K^{-1} @ [x, y, depth(x, y)] where @ denotes matrix multiplication.
As I mentioned above, in order to do this, you will need the depth of pixel x, y. This can be obtain for instance using an RGB-D camera.
If you do not have access to an RGB-D camera, you might want to look into Structure-from-Motion - i.e., building a 3D model from a sequence of images.
from pycolmap.
@mihaidusmanu thank you! I have managed to extract the 3D points using the depth image.
import cv2
# camera matrix
mtx = [8.6767822265625000e+02, 0., 4.8268579101562500e+02, 0.,
8.6760119628906250e+02, 2.7117459106445312e+02, 0., 0., 1.]
# parse calib matrix
fx = mtx[0]
cx = mtx[2]
fy = mtx[4]
cy = mtx[5]
depth_img = cv2.imread("depth.png", cv2.IMREAD_ANYDEPTH)
depth_row, depth_column = depth_img.shape
# to store 3d points
pts3d = []
for i in range(depth_row):
for j in range(depth_column):
# retrieve depth value
depth = depth_img[i][j]
x, y = i, j
if depth > 0.0:
x3D = (x - cx) * depth / fx
y3D = (y - cy) * depth / fy
z3D = depth
# populate 3d points array
pts3d.append([x3D, y3D, z3D])
else:
pts3d.append([-1, -1, -1])
I have key points from SIFT. Any idea on "points2D" which is the first parameter?
from pycolmap.
I would like to obtain the absolute pose of an image given a point cloud. I'm still having a hard time trying to get the perfect absolute position :( Is there any example code?
from pycolmap.
I have implemented the first portion so I have points2D
from the image and points3D
from the depth image.
In terms of params
, I have used the values of the image corresponding to the points2D
. (fx,cx,cy)
values.
The resultant absolute pose has a pose estimation error of about 50-70cm. am I doing it properly?
from pycolmap.
hey can you provide the solution you implemented? I am having the same trouble as you
from pycolmap.
Related Issues (20)
- AttributeError: module 'pycolmap' has no attribute 'Database' HOT 2
- Trying to install pycolmap either via the pip or via source not working HOT 4
- camera.img_from_cam not working HOT 1
- May I ask about the changes in the new version? HOT 1
- AttributeError: module 'pycolmap' has no attribute 'patch_match_stereo' HOT 2
- Type hints for python? HOT 4
- Are Point3D and Image 0 indexed? HOT 2
- Error when importing pycolmap HOT 5
- How to set up scaling sparse models to realistic sizes? HOT 1
- When should a class be bound with shared_ptr as its holder? HOT 2
- pycolmap build from source error HOT 1
- estimate_triangulation result not equal to incremental_mapper result HOT 2
- absolute_pose_estimation HOT 1
- using pycolmap align two scene But either the exact same image is not captured (no overlapping images), or only part of the images overlap
- Question about pycolmap.incremental_mapping HOT 3
- Bind Iterative Global SfM HOT 2
- error in absolute_pose_estimation
- Script to export our own wheel HOT 5
- How to resolve conflicts between pycolmap and pytorch HOT 3
- About the function pycolmap.triangulate_points HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pycolmap.