More information about the proper setup is needed

[kurshakuz]

More information about the proper setup is needed

[zhangbaozhe]

Is that code available open source? I would want to take a look, if possible of course

Sorry, the code is not available. The main idea however is to first get the depth information and then listen to the click event in each frame of the openCV iteration, and then change the tracker pointers.

[zhangbaozhe]

Hi.

If you have read this code in the ICG official repo here, you will find that my code is actually follow its style, except for changing the camera interface to ROS messages and putting the main tracking process in a `usual ROS while loop' (the RunTrackerProcessOneFrame method).

The camera I use is Intel Realsense D435. Before I run the icg_test_node, I first run a camera launch to start the camera using this command

roslaunch realsense2_camera rs_aligned_depth.launch 

Then after you start the icg_test_node, you will see a OpenCV GUI window of the color image. Right now, the function of the window is almost the same as that in the ICG repo. To use it for tracking, you may want to see this code in advance, here. You can see that the window is listening for you key strokes. If you press 'd', it will start detection (I think it just renders the predefined pose in the config yaml on the screen). Then if you press 't', it will start track. And in my code, the case would be that the triangular shape should be start moving around (tracking started).

The tracking procedure is following a state-machine style. So the message in the start_tracking topic is sent to this function here. If it is true, the message sets the execute_detection_ and start_tracking_ variables to be true, which should be the same as pressing 'd' and 't'. I just tested the code, and this feature might have some bugs. I will check it later. But the idea is OK, I had tested it earlier, and it was fine.

As for pose, actually, as you can see in the code, it is not published yet. However, you can write some output lines to print the temp_transform. If the triangle moves around, the value of this transform should change.

This repo is still under development. Feel free to contribute!

[zhangbaozhe]

Bug fixed. Now you can use the start_tracking topic.

[kurshakuz]

Thanks! That makes it clear now.

[kurshakuz]

Hey @zhangbaozhe I have one more question that I think you could have thought of. I am not proficient in C++, but how do you think is hard to replace the static detector part with code that can update the pose depending on a trigger? Assume we have a node that is able to quantify pose drift of the object, and send the trigger with a new pose estimate to the detector part of ICG. Is updating the SetUp method would be sufficient?

[zhangbaozhe]

I am not sure what you meant. I suppose you meant that there can be another node running like similar pose estimation algorithm and sending out the pose information to the ICG node for ICG to update its own estimation if there is pose tracking loss (or drifting).

Actually, in the project I participated in earlier using the ICG package, we tried to use the clicking event in the openCV window to update the detection manually and then start tracking (but that code is poorly written). I think ICG is a pose tracking method, which means the tracking accuracy highly depends on its initial given pose (of course you can set a higher search value to let the result converge). Maybe for your project, you may consider using a detection method first (e.g., YOLO, although YOLO does not provide you with the orientation info) and then give that value to ICG to track.

By the way, for this library, I prefer to change the code of its own as little as possible and only add interfaces, e.g., friend functions to change the pointers inside the classes. So as for your last suggestion about updating the pose estimation with a trigger, I would add a switch for whether to use external estimation, and if yes add a function that can quantify the error and update the pose in ICG if the error is larger than some threshold. But another problem immediately occurs, how can you guarantee the other pose estimation method is accurate? Maybe here it is more like a system (try EKF?) problem rather than the problem for ICG.

[kurshakuz]

I am not sure what you meant. I suppose you meant that there can be another node running like similar pose estimation algorithm and sending out the pose information to the ICG node for ICG to update its own estimation if there is pose tracking loss (or drifting).

I rather meant to get the tracking/drift error from ICG directly and send trigger to the pose re-initialization node that has to be in a separate package as it is not related to ICG. However, this idea makes sense as well.

Actually, in the project I participated in earlier using the ICG package, we tried to use the clicking event in the openCV window to update the detection manually and then start tracking (but that code is poorly written).

Is that code available open source? I would want to take a look, if possible of course

I think ICG is a pose tracking method, which means the tracking accuracy highly depends on its initial given pose (of course you can set a higher search value to let the result converge). Maybe for your project, you may consider using a detection method first (e.g., YOLO, although YOLO does not provide you with the orientation info) and then give that value to ICG to track.

Yeah makes sense. Instead of using DL detection methods, I would assume using point cloud registration methods (using ICP, RANSAC, TEASER++) would be preferable, as we already have 3D ply model. Additionally, it does not require any training.

By the way, for this library, I prefer to change the code of its own as little as possible and only add interfaces, e.g., friend functions to change the pointers inside the classes. So as for your last suggestion about updating the pose estimation with a trigger, I would add a switch for whether to use external estimation, and if yes add a function that can quantify the error and update the pose in ICG if the error is larger than some threshold. But another problem immediately occurs, how can you guarantee the other pose estimation method is accurate? Maybe here it is more like a system (try EKF?) problem rather than the problem for ICG.

Agreed, there is no 100% perfect solution for that. TEASER++ for example provide good registration capabilities, but is not able to track object over time. Combination of those two could make sense. I will try looking into it