lardemua/atom

Calibration tools for multi-sensor, multi-modal robotic systems

ATOM Calibration

A Calibration Framework using the
Atomic Transformations Optimization Method

ATOM is a set of calibration tools for multi-sensor, multi-modal, robotic systems, based on the optimization of atomic transformations as provided by a ROS based robot description. Moreover, ATOM provides several scripts to facilitate all the steps of a calibration procedure.

For instructions on how to install and use, check the documentation:

https://lardemua.github.io/atom_documentation/

and check these examples.

Also, you can take a look at the ATOM youtube playlist.

Support

If this work is helpful for you please cite our paper:

• Oliveira, M., E. Pedrosa, A. Aguiar, D. Rato, F. Santos, P. Dias, V. Santos, ATOM: A general calibration framework for multi-modal, multi-sensor systems, Expert Systems with Applications (2022), 118000, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2022.118000. Bibtex.
  

or any other that is more adequate:

• Gomes, M. M. Oliveira, V. Santos, ATOM Calibration Framework: Interaction and Visualization Functionalities, Sensors (2023), 23, 936. https://doi.org/10.3390/s23020936. Bibtex.
  

• Rato, D., M. Oliveira, V. Santos, M. Gomes, A. Sappa, A sensor-to-pattern calibration framework for multi-modal industrial collaborative cells, Journal of Manufacturing Systems (2022), Volume 64, Pages 497-507, ISSN 0278-6125, https://doi.org/10.1016/j.jmsy.2022.07.006. Bibtex.
  

• Pedrosa, E., M. Oliveira, N. Lau, and V. Santos, A General Approach to Hand–Eye Calibration Through the Optimization of Atomic Transformations, IEEE Transactions on Robotics (2021) pp. 1–15, DOI: https://doi.org/10.1109/TRO.2021.3062306, 2021. Bibtex.
  

• Aguiar, A., M. Oliveira, E. Pedrosa, and F. Santos, A Camera to LiDAR calibration approach through the Optimization of Atomic Transformations, Expert Systems with Applications (2021) p. 114894, ISSN: 0957-4174, DOI: https://doi.org/10.1016/j.eswa.2021.114894, 2021. Bibtex.
  

• Oliveira, M., A. Castro, T. Madeira, E. Pedrosa, P. Dias, and V. Santos, A ROS framework for the extrinsic calibration of intelligent vehicles: A multi-sensor, multi-modal approach, Robotics and Autonomous Systems (2020) p. 103558, ISSN: 0921-8890, DOI: https://doi.org/10.1016/j.robot.2020.103558, 2020. Bibtex.
  

Contributors

• Miguel Riem Oliveira - University of Aveiro
• Afonso Castro - University of Aveiro
• Eurico Pedrosa - University of Aveiro
• Tiago Madeira - University of Aveiro
• André Aguiar - INESC TEC
• Daniela Rato - University of Aveiro
• Manuel Gomes - University of Aveiro

Maintainers

• Miguel Riem Oliveira - University of Aveiro
• Manuel Gomes - University of Aveiro

atom's Issues

Collect a bag file with the UR10 for calibrating camera w.r.t. robot

Create a tweked robot_description param

The goal is to have our code read through a parameter (lets called initial_robot_description).

Then, we tweak this by adding the sufix "_initial" to all links which are listed in the sensors.

But I am not sure this is really needed because of how the new tf handles the fixed transforms (probably as static transforms),

Detected and stored data is the same for different cameras

The collect_and_label_data.py stores the images and the detected chess corners pixel indexes of the top right camera for both camera sensors. There is double equal information when it should be distinct data for distinct cameras.

Decide which code formulation to use

After simlutaneously doing #7, we need to define which code we are going to use, or if some hybrid. You make that decision. We must know this in order to continue the development in a sinlge code.

Image from frontal camera is very slow rate and seems like it is not debayered

Merge 6DOF and Menu marker python test

Calibrate frontal camera

follow this

http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration

Implement Lidar data cost function

How to configure catkin build

Hi @eupedrosa ,

can you tell me what must be done to configure catkin build?

Thanks,
Miguel

Drawing strange lines in Optimization utils

The 3d drawing of the optimization has some weird lines.

Here's an example:

Consider dynamic transforms for the pre and post transfoms

Hi @afonsocastro ,

this is related to what Jorge Almeida said.

I realized that there is yet another problem. Some sensors are mounted on dynamic kinematic chains. For example the cameras on the PR2 are on top of a moving head.

I think the Sensor class must be re-engineered to support this. The good thing is that it would also solve the problem Jorge Mentioned.

Lets talk about it.

Transforms dict should be within the collections key value

This change implies an adaption of collect_and_label_data.py. Also in main.py, objective_function.py, results_visualization.py, getter_and_setters.py from OptimizationUtils/test/sensor_pose_json_v2, stereocalib_v2.py. Anywhere else?

Create a simple test for atlascar calibration optimization

Get inspiration from:

https://github.com/miguelriemoliveira/OptimizationUtils

Create a -configuration_file command line parameter

This would be useful to avoid a lot of command line arguments. We could use a json config file instead.

Show atlacar cad in rviz

Data collector save image - path should be relative to dataset folder

Otherwise it is not portable ...

Create a data labeller

Dangerous piece of code

class DataCollectorAndLabeler:
    def __init__(self, world_link, output_folder, server, menu_handler, marker_size, chess_numx, chess_numy):
        if os.path.exists(output_folder):
                shutil.rmtree(output_folder)  # Delete old folder

What would you do if a third party software did this on your computer without explicit permission? 🤔

New atlascar bag file contains incorrect stuff

I guess we need a new bag file.

Solve warning "xacro.py is deprecated; please use xacro instead"

Sensors Pose First Guess not loaded

If we launch roslaunch interactive_calibration atlascar2_calibration.launch and then rosrun interactive_calibration create_first_guess.py -w base_link, we can move the sensors freely and see directly the changes of the point clouds and the images (as it was expected).

Then we can save our best eye-belief config for the sensors pose. The problem is when I try to re-run the launch file with the read_first_guess argument, rviz opens and the sensors pose are exactly the same as in the original urdf robot description.
I found that the problem is related to the bag file that is running because if I test this first guess process without running any bag file it all goes well...
I think that the bag must be publishing some tf's about the sensors pose, but I am not sure of this. @miguelriemoliveira, can you help me figure out what is wrong here?

PR2 xacro: 'type' attribute and the 'compensator' tag are unknown in some links

low priority for this one ...

Add variables in sensor xml to represent opt, pre and post transforms

@afonsocastro FYI.

Since you added the functionality of reading the xml tag sensor in the urdfdom_py, I think we can also add the elements we required for out calibration.

Will add them and change the code accordingly.

Create a json file from data collected during a calibration acquisition

rviz crashes when changing desktops

When running

roslaunch interactive_marker_test pr2_calibration.launch

whenever I change desktops rviz crashes with the following error:

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
  what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument
================================================================================REQUIRED process [rviz-3] has died!
process has died [pid 13431, exit code -6, cmd /opt/ros/melodic/lib/rviz/rviz -d /home/mike/catkin_ws/src/AtlasCarCalibration/interactive_marker_test/calibrations/pr2/config.rviz __name:=rviz __log:=/home/mike/.ros/log/c227165a-7bcb-11e9-ba2d-0028f80a63ec/rviz-3.log].
log file: /home/mike/.ros/log/c227165a-7bcb-11e9-ba2d-0028f80a63ec/rviz-3*.log
Initiating shutdown!
================================================================================

Use PR2 for complete tf trees

Store initial guess, pre and post transform

@fyi @afonsocastro

Lidar data labelling

Hi @miguelriemoliveira ,
I tested to label the lidar data as you asked and it worked! For now, the labeller allows NaN in range measurements, but we have the problem of only recording some ranges between all the important points. More than that, the size of the cluster collected change in every capture and for each laser...

Enhance documentation of the creation of a new optimization

Use a urdf parser for reading the robo description

That should be the best approach to reading the information and fiding out which sensors the robot has...

Sensors first guess's orientation isn't correctly saved

for joint in robot.joints: for sensor in robot.sensors: if sensor.parent == joint.child: optimization_parent_link = joint.parent for mp in marker_poses: (trans, rot) = listener2.lookupTransform(optimization_parent_link, mp.child_frame_id,rospy.Time(0)) if joint.child + "_first_guess" == mp.child_frame_id: joint.origin.xyz[0] = trans[0] joint.origin.xyz[1] = trans[1] joint.origin.xyz[2] = trans[2] joint.origin.rpy[0] = rot[0] joint.origin.rpy[1] = rot[1] joint.origin.rpy[2] = rot[2]

Reset marker poses

Hi @afonsocastro,

we should have an additional option in the rviz menu to reset all marker poses to the original position. Can you implement?

Error parsing PR2 description

@hi @afonsocastro ,

I have an error when I launch create_first_guess.py using the PR2.

Unknown attribute "type" in /robot[@name='pr2']/link[@name='base_laser_link']
Unknown attribute "name" in /robot[@name='pr2']/link[@name='torso_lift_link']/collision[1]
Unknown tag "simulated_actuated_joint" in /robot[@name='pr2']/transmission[@name='torso_lift_trans']
Unknown attribute "type" in /robot[@name='pr2']/link[@name='wide_stereo_optical_frame']
Unknown attribute "type" in /robot[@name='pr2']/link[@name='narrow_stereo_optical_frame']
Unknown attribute "type" in /robot[@name='pr2']/link[@name='laser_tilt_link']
Unknown tag "compensator" in /robot[@name='pr2']/transmission[@name='r_shoulder_pan_trans']
Unknown tag "compensator" in /robot[@name='pr2']/transmission[@name='r_shoulder_lift_trans']
Traceback (most recent call last):
  File "/home/mike/catkin_ws/src/AtlasCarCalibration/interactive_marker_test/src/create_first_guess.py", line 102, in <module>
    xml_robot = URDF.from_parameter_server()
  File "/home/mike/catkin_ws/src/urdf_parser_py/src/urdf_parser_py/urdf.py", line 667, in from_parameter_server
    return cls.from_xml_string(rospy.get_param(key))
  File "/home/mike/catkin_ws/src/urdf_parser_py/src/urdf_parser_py/xml_reflection/core.py", line 612, in from_xml_string

Do you have the same problem?

add input parameter for storing the common reference frame

Better 3D plotting?

Hi @afonsocastro , @tiagomfmadeira and @eupedrosa ,

we are using matplotlib for drawing the 3D graphs, but the quality of the figures is not very good.
Here's an example:

I looked around and found this

do you know any good alternatives for 3D plotting with python?

New command line argument to show images for each camera

Hi @afonsocastro @tiagomfmadeira @eupedrosa

for large optimizations (e.g. 30+ collections) it is not possible to show all camera images. For that reason I added an argument -si or --show_images to the optimization utils sensor_pose_v2/main.py which must be given if you want to see the images.

Example:

test/sensor_pose_json_v2/main.py -json ~/datasets/calib_bad_first_guess/data_collected.json -cradius .5 -csize 0.101 -cnumx 9 -cnumy 6 -vo -si

not the default behavior now is not to see the images...

Why the f*%$k is the car model misplaced?

This occurs on my system but not in @afonsocastro 's system. They are git synced so I am really not sure what is happening.

@afonsocastro , can you test it on a third system to see if it is your system or mine that is out of the norm?

Create or get a bag file of an autonomous vehicle (or Instrumented Car)

Related to #21

Create new launch file for each robot model

And add each rviz config

Run an optimization with only the cameras to make sure we did nothing wrong in last Tuesday's changes.

@afonsocastro , can you do this?

Record a bag file from the atlascar so it is possible to watch sensor data while creating the first guess

Not sure we will need #9 for this functionality or not. But we must record a bag file anyway.

@afonsocastro , lets do it on monday afternoon? Meet at 14h?

Create or get a bag file of a PR2

Related to #15

Tracking of chessboard in 3D

To use the asus xtion check https://wiki.ros.org/openni2_launch

Devise initial camera to camera optimization

Hi @afonsocastro ,

I think we are now very close to point where we will start with the optimizations.

So you have a new task which is to study the examples in

https://github.com/miguelriemoliveira/OptimizationUtils/tree/master/test

in particular these ones

simple optimization that changes image colors
https://github.com/miguelriemoliveira/OptimizationUtils/blob/master/test/global_color_balancing_oc_dataset.py

camera to N cameras optimization
https://github.com/miguelriemoliveira/OptimizationUtils/blob/master/test/camera_pose_oc_dataset.py

So start studying it and try to set up here an ordered list of stuff you must do to implement the optimization in your case.

Data Collector change structure of dictionaries containing collected data

Reference Optimization utils in the readme

@eupedrosa was trying to test and could not find a reference to the OptimizationUtils. Should be there.

Reorganize code to make it easier to make changes ...

Code Reorganization

This is already done in commit d07f214

Followed best practices from:

http://wiki.ros.org/rospy_tutorials/Tutorials/Makefile

and

http://wiki.ros.org/PyStyleGuide

Note on github about installing flycap

Hi @afonsocastro, (@tiagomfmadeira , @eupedrosa )

flycap is needed for this package to compile. Add a note on the readme about that please.

Create a graph of the pre, calibration and post transforms

@afonsocastro wating for #15 to continue ...