Comments (34)
Hi,
I need to find camera matrix (3x3) and distort coefficient (1x5) inorder to find pose estatimation using opencv of known pattern. But in SNxxxx.conf file I can find only fx, fy, cx, cy, k1 & k2.
Camera Matrix 3x3:
| fx 0 cx |
| 0 fy cy |
| 0 0 1 |
fx : focal length in direction of camera's x axis
fy : focal length in direction of camera's y axis
cx,cy : Principal point in image
Distortion Coefficient (1x5):
[k1 k2 p1 p2 k3]
k1,k2,k3 : Radial distortion coefficients
p1, p2 : Tangential distortion coefficinets
k3 - set to zero (only applicable for fish-eyed lens)
-
How can I get distortion coefficient p1, p2?
-
If we do camera calibration using opencv "calibrateCamera(obj_pts, image_pts, Size, camera_matrix, distoriton_coeff, rot_temp, trans_temp, CV_CALIB_FIX_K3);" we can get camera matrix and distortion coefficients.
Can we do this calibration with the images we get from zed.retrieveImage - left image (this Image is rectified) ? If i use opencv videocapture to grab ZED camera image I am getting image of size 376x1344. How to get images of size 2K, FHD & HD for calibration?
Will the camera matrix determined with rectified image be same with SNxxxx.conf file?
from zed-ros-wrapper.
Hi Aduljadin,
I would like to process images saved from ZED Explorer. I found camera parameters from SN####.conf file. How is stereo rotation-translation matrix computed from RX, RZ, CV, and Baseline?
from zed-ros-wrapper.
Hi @chuong,
The rotation (in rotation vector form, convertible into matrix with Rodrigues formula) are the following :
Rx = RX
(in rad),
Ry = CV
,
Rz = RZ
.
The translations are set to 0 except of course for Tx :
Tx = - baseline
,
Ty = 0
,
Tz = 0
.
However please note that these values are refined by the autocalibration process to achieve higher accuracy with the ZED SDK.
from zed-ros-wrapper.
Hi,
The images outputting from the ZED SDK are already rectified. So you can ignore the calibration process, it's already taking care of.
However if you want to recalibrate you can use our ZED Calibration tool (easiest solution) or other software (openCV, ROS stereo camera calibration node...) but in the later, you will need to provide unrectified images which are currently not accessible using the ZED SDK. So you will need to code an other ROS node using VideoCapture or something similar.
from zed-ros-wrapper.
I just found out the calibration file that I downloaded manually from this link "http://calib.stereolabs.com/?SN=NNNN" (I got the same result with 0000NNNN) which NNNN is my camera's serial number is totally different than the one in settings folder. Here are the parameters for VGA from the downloaded file
[LEFT_CAM_VGA]
fx=699.417
fy=699.417
cx=297.493
cy=243.168
k1=-0.171585
k2=0.0253922
[RIGHT_CAM_VGA]
fx=700.648
fy=700.648
cx=350.405
cy=259.796
k1=-0.173575
k2=0.0264623
and here are the parameters from the file in the settings folder
[LEFT_CAM_VGA]
cx=323.747
cy=188.584
fx=349.707
fy=349.707
k1=-0.171585
k2=0.0253922
[RIGHT_CAM_VGA]
cx=350.203
cy=196.898
fx=350.325
fy=350.325
k1=-0.173575
k2=0.0264623
Would somebody please explain why the two files are different?
from zed-ros-wrapper.
Hi @aseyfi
The difference is due to the firmware. The new firmware (1142) introduced the wide VGA. However, your camera has been factory calibrated with the previous firmware (922, 4:3 VGA).
The setting file download / conversion is handled by the ZED Explorer (with command line option --dc or GUI) but not yet with https://calib.stereolabs.com since we can't determine the firmware version in use.
So the parameters from your settings folder (the second) are the correct ones.
from zed-ros-wrapper.
Hi @adujardin ,
Thanks for the info. Is stereo rotation matrix equal to Rx*Ry*Rz
or Rz*Ry*Rx
?
I calculated stereo translation vector and rotation matrix for HD resolution as:
T = np.array([-BaseLine, 0, 0])
Rz, _ = cv2.Rodrigues(np.array([0, 0, RZ_HD]))
Ry, _ = cv2.Rodrigues(np.array([0, CV_HD, 0]))
Rx, _ = cv2.Rodrigues(np.array([RX_HD, 0, 0]))
R = np.dot(Rz, np.dot(Ry, Rx)) # Rz*Ry*Rx
I tried bothRz*Ry*Rx
and Rx*Ry*Rz
for R, but epipolar lines on stereo rectified images are off about 5 pixels between same feature points on stereo images. I got the parameters after recalibrate the camera with ZED Calibration tool. Are the parameters in SN####.conf file as good as stereo calibration using OpenCV with a normal chessboard?
How can I get refined camera parameters from autocalibration process?
from zed-ros-wrapper.
Sorry. I made a mistake by undistorting images before stereo rectification. It works correctly now.
It seems either Rz_Ry_Rx or Rx_Ry_Rz work fine for R.
from zed-ros-wrapper.
Thank you @adujardin . I am using Matlab to open ZED as a regular webcam. The only possible resolution is 2560x720 which is actually two 1280x720 images for the left and right cameras. I would like to process the images in VGA, exactly the same size as ZED ROS wrapper is publishing (672x376). Should I just resize the whole 1280x720 image to 672x376? Are there any other steps involved?
from zed-ros-wrapper.
hi @aseyfi ,
take a look at the Stereolabs Blog, at the end of the page there is a snippet explaining how to grab the ZED images.
The line:
zed.Resolution = zed.AvailableResolutions{1};
let you choose the resolution from the list 'AvailableResolutions' displayed by
zed = webcam('ZED')
from zed-ros-wrapper.
Thank You @P-yver . In that example we can see "AvailableResolutions: {1x4 cell}" which means there are four different resolution options and and they are using the first one (zed.Resolution = zed.AvailableResolutions{1};). In my case, I have only one option (AvailableResolutions: {'2560x720'}). I am using Mac and there is no ZED SDK for Mac. Do I need to install the SDK if I need to have access to all of the possible resolutions?
from zed-ros-wrapper.
Hi, I am using the "zed_ros_wrapper" to publish HD images which they are undistorted and rectified (output of ZED SDK). I fixed the camera facing to a scene and saved one frame from the laft camera and one from the right. then I selected one point in the scene. Assuming it is located on [col_l, row_l] and [col_r, row_r] in the left and right images respectively, what are the calculations based on [col_l, row_l], [col_r, row_r] and the data in the calibration file to find 3D location of that point in the world?
from zed-ros-wrapper.
Hi @aseyfi ,
Based on our convention you have :
disparity = col_r - col_l // should be negative
then the 3D point (X Y Z):
Z = (fx_l * Baseline) / (-disparity)
X = (col_l - cx_l) * Z / fx_l
Y = (row_l - cy_l) * Z / fy_l
Have you fixed your problem with the desired resolution on Mac ? I have not yet tested it (we currently don't support Mac).
from zed-ros-wrapper.
Thank you @P-yver . No, actually I installed Matlab on Ubuntu 14.04 and I get this output. The available resolution is different, but I still have only one option
cam=webcam
cam =
webcam with properties:
Name: 'ZED'
Resolution: '4416x1242'
AvailableResolutions: {'4416x1242'}
based on this instructions (https://www.stereolabs.com/blog/index.php/2015/07/03/tutorial-3-zed-and-matlab/) I am supposed to see this output:
cam =
webcam with properties:
Name: 'ZED'
Resolution: '2560x720'
AvailableResolutions: {1x4 cell}
WhiteBalanceMode: 'auto'
Sharpness: 4
Saturation: 5
Hue: 0
Gain: 4
WhiteBalance: 4600
Contrast: 4
Brightness: 4
Exposure: 2
ExposureMode: 'auto'
EDIT: My Matlab version is R2016a
from zed-ros-wrapper.
@P-yver , I used the formulas you provided to estimate 3D location of two points in the world. I fixed my camera about 90cm away from a wall and marked two points with 80cm distance on the wall. Then I used zed_ros_wrapper to capture WVGA images from the left and right cameras. Using Matlab, I clicked on the points on the left and right images (as accurate as I could) and found the [row, col] values of each point. Then I used the provided formulas to calculate (X,Y,Z) of each point and calculated the distance between the two points afterwards. From my calculations, I estimated the distance between the two pint 77.5cm. I carried out a similar test for two other points with different distances from the camera too. The actual distance between two points was 6.1m and I estimated it to be 4.86m (1.24m error). Is this the accuracy I should expect from ZED?
I have a question about rectification too: We know that after un-distorting and rectifying the images, they will warped and we have to crop a square out of the warped images. Depending on the rectangle that we select, the camera matrix would be different. Does ZED crop/scale the rectified images in a way that the camera matrix remains exactly as it was estimated during calibration?
from zed-ros-wrapper.
Hi @aseyfi ,
The camera parameters given by the .conf file (or shown by the ZED Explorer) are the initial values. They are changed by the auto-calibration step performed by the SDK during the initialization. To retrieve the parameters corresponding (modified by the rectification) you have to get those parameters with 'left_cam_info_topic'.
The disparity is not linear (it decreases rapidly with the distance) and the sub pixel accuracy is required to achieve realistic measures, click on images is not appropriate to benchmark the process.
The ZED Depth Viewer display the depth when you click on the disparity/depth/left image as well as the sample ZED with OpenCV. Why did you use the ros wrapper to capture images ?
from zed-ros-wrapper.
Hi @P-yver , Thank you very much for your comment, it was really helpful and I was able to get the rectified image's camera matrix. I was initially using zed_ros_wrapper to publish data for another software in ROS. Then I was trying to go through the steps and understanding stereo vision theory, so I was just more comfortable using this package compared to OpenCV.
from zed-ros-wrapper.
@P-yver Are the snapshots captured by ZED Explorer raw images?
from zed-ros-wrapper.
Hi @aseyfi ,
Yes, the snapshots captured by ZED Explorer are raw. There is no processing on ZED Explorer, only visualization. The images are therefore unrectified.
from zed-ros-wrapper.
Thank you @P-yver . In you previous comment you mentioned auto-calibration:
"They are changed by the auto-calibration step performed by the SDK during the initialization."
Would you please let me know what the auto-calibration is? Can it change the camera matrix of the rectified images every time it is being initialized?
Also, I am still having problem to access all of the possible resolutions of ZED in Matlab R2016a and Ubuntu 14.04, and the only resolution that I can have access is '4416x1242'. Does ZED support Matlab R2016a on Ubuntu 14.04 and do you see the same issue?
Thank you,
from zed-ros-wrapper.
Hello, I printed out the data published on "/right/camera_info" using "rostopic echo" command and here is part of the published data
height: 376
width: 672
distortion_model: plumb_bob
D: [0.0, 0.0, 0.0, 0.0, 0.0]
K: [337.1528015136719, 0.0, 349.25897216796875, 0.0, 337.1528015136719, 197.14279174804688, 0.0, 0.0, 1.0]
R: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
P: [337.1528015136719, 0.0, 349.25897216796875, -0.04045833647251129, 0.0, 337.1528015136719, 197.14279174804688, 0.0, 0.0, 0.0, 1.0, 0.0]
I think the fourth element of P is representing fx * baseline. Using the baseline=0.12m and fx=337.15, I expect the number to be (337.15*0.12)=40.46, but the published number is 0.04045. Why is the published number scaled?
from zed-ros-wrapper.
Hi, I installed zed sdk v2.0.0 & cuda 8 (Laptop dell precision 7710 - nvidia graphic card quadro M5000M, windows 8.1 pro). I have updated zed camera firmware to zed_fw_v1142_spi through ZED Explorer.
When run Depth viewer I am get "Unable to open the ZED. Error Code: calibration Filte not available" message.
I downloaded SN1947.conf file from "https://www.stereolabs.com/developers/calib/"pasted "C:\Users\Elgi\AppData\Roaming\Stereolabs\settings" folder. Also, I generated conf file using ZED Calibration. But, Still the same error persist.
Can you suggest how to overcome this?
from zed-ros-wrapper.
Hi, arkrobo. I had same problem.
"https://www.stereolabs.com/developers/calib/" may be out of date .
You should try self calibration with ..../zed/tools/ZED Calibration.
Before calibrate, remove or rename the old SNxxxx.conf.
Then new calibration file is made in setting/NNNN.conf and solved in my case.
Thanks.
from zed-ros-wrapper.
@arkrobo I had same problem too. I think there is something unexpected happened when I run /ZED/tools/ZED Depth Viewer. After trying ../tools/ZED Calibration, I can run every .exe in ..ZED/sample/bin. But if I run ../tools/ZED Depth Viewer and change the resolution into VGA@100, the app will throw an error. And it will download SNxxxx.conf automatically and replace the SNxxxx.conf generated by ../ZED Calibration, which will lead to every .exe in ZED/sample/bin not working, and throw the "Unable to open the ZED. Error Code: calibration Filte not available".
So please check if the SNxxxx.conf file has been changed automatically if you meet this error. My English is not very good, hope can be helpful.
from zed-ros-wrapper.
Thanks S- Shiozawa & axing0805091,
The generated SNxxxx.conf file is overwritten by online conf file. fixed the issue by replacing the generated file.
from zed-ros-wrapper.
Hello @adujardin,
The rotation (in rotation vector form, convertible into matrix with Rodrigues formula) are the following :
Rx = RX (in rad),
Ry = CV,
Rz = RZ.
Can you please clarify which of the following options is true:
- Rotational matrix = Rodrigues([Rx, Ry, Rz]))
or - Rotational matrix = Rodrigues([Rx, 0, 0]))*Rodrigues([0, Ry, 0]))*Rodrigues([0, 0, Rz]))
or - Rotational matrix = Rodrigues([0, 0, Rz]))*Rodrigues([0, Ry, 0]))*Rodrigues([Rx, 0, 0]))
Thanks.
from zed-ros-wrapper.
The first one
- Rotational matrix = Rodrigues([Rx, Ry, Rz]).
So, for instance something like this using OpenCV Rodrigues :
cv::Rodrigues([Rx, Ry, Rz] /*in*/, RotMatrix3x3 /*out*/)
from zed-ros-wrapper.
Hi there, followed your conversation, a question i have, i am trying to calibrate a DSLR lens to the left eye (cam) from the ZED per custom OCV script in order to displace me 4K video later to achieve a 6DOF in VR. the depth map is aligned with the left camera from the ZED and i have managed to rig the ZED above the physical lens and got a pretty decent match per hand, basically fine tuning (warping and un-distorting) my live video feed from the DSLR in Nuke but there is sure a much better way to do this, at least optically more precise. Can someone point me in the right direction? Thank you in advance, Mark
from zed-ros-wrapper.
Hi @adujardin ,
The rotation matrix called Rotm
calculated by
cv::Rodrigues([Rx, Ry, Rz] /*in*/, RotMatrix3x3 /*out*/)
.
I want to know if the Rotm
is the rotation from left camera to right camera or from right camera to left camera.
And is the camera coordinate system in right-handed,y-down coordinate system.
Thanks.
from zed-ros-wrapper.
@adujardin How can we make the ZED camera use calibration parameters obtained by ROS ?
from zed-ros-wrapper.
Can we get the cx and cy (principal points of the camera) using the ZED Camera API? Basically, by calling some function on left_cam or right_cam?
from zed-ros-wrapper.
@yaminiMathur please do not post questions on closed issues.
Create a new issue instead
from zed-ros-wrapper.
The first one
- Rotational matrix = Rodrigues([Rx, Ry, Rz]).
So, for instance something like this using OpenCV Rodrigues :
cv::Rodrigues([Rx, Ry, Rz] /*in*/, RotMatrix3x3 /*out*/)
Hello, I have a question about this stereo parameter. Is this the transform matrix between left camera frame and right camera frame? Or is it between optical frame?
from zed-ros-wrapper.
@yaminiMathur please do not post questions on closed issues. Create a new issue instead
from zed-ros-wrapper.
Related Issues (20)
- Object Detection on Rosbag Data HOT 1
- Depth image msg contains only zero values HOT 14
- ZED ros wrapper really slow on orin nano! HOT 3
- resolution mismatch for ros noetic HOT 1
- ZED streaming on long range ethernet HOT 1
- Filtered Depth Not published in ROS topic HOT 2
- ZED camera point cloud inaccuracy: white wall HOT 1
- Cannot catkin build with SDK 4.1 HOT 4
- Positional tracking GEN_2 strange behaviors: covariance values and randomly stopping publishing messages HOT 4
- ZED-ROS-ERROR:Invalid calibration file HOT 3
- Documentation update, openni_depth_mode is located in a yaml file not a launch file anymore
- Camera internal parameter problem
- Can´t catkin_make in Win10 ROS noetic HOT 1
- roslaunch hangs on "* Waiting for valid static transformations..." HOT 2
- separate downscale for depth and color (was available before) HOT 2
- rosbag recorded imu and image sampling intervals are not stable HOT 1
- Zed X Position tracking does not move when imu_fusion is True HOT 1
- Positional Tracking in ROS wrapper ZED2i
- Enable Dynamic Resolution Change for ZED Cameras in ROS2 Wrapper HOT 1
- Compilation fails with catkin_make -DCMAKE_BUILD_TYPE=Release HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zed-ros-wrapper.