Reprojection yielded 0 valid points :(

[sdmiller]

Reprojection yielded 0 valid points :(

[sdmiller]

Sorry for the delay! Been traveling.

Yes, so it looks to me like this is failing during the "Reprojection" phase. That's something I added strictly to the integrate script, since a bunch of people were working with unorganized clouds.

Basically, the TSDF algorithm assumes it is given a series of organized clouds (such that cloud(u,v) corresponds to pixel (u,v)). Since most average users were not happening to pass in clouds of this variety, I added a little trick to the script which synthetically re-roganized the cloud. It does this using the intrinsics of the sensor (focal length, center pixel) to backproject the points into pixel space. By default we use the intrinsics of a standard kinect sensor.

That leads to a few things:

• If your data is already organized, you can pass in the "--organized" flag to tell the script not to do this reprojection. BUT
• Regardless of whether it is organized, if reprojection is completely failing here it probably means the camera intrinsics (--fx,-- fy, --cx, --cy, --width, --height) are drastically different from what we expect them to be. You'll want those to be correct to get reasonable results from the TSDF.
• I'm still shocked that reprojection worked in literally 0 cases. This indicates to me that the clouds are extremely different from what we are expecting. I'd love to take a look at one and see.

When in doubt, I'd suggest hopping into integrate.cpp and looking at the for loop where it reprojects points. A printout or two there could help dramatically.

[sdmiller]

Also worth noting: if your data is organized and you don't know the intrinsics, you can use my handy little get_intrinsics tool (src/prog/get_intrinsics.cpp) to get a reasonably good estimate

[eric-schleicher]

Thanks for taking the time to response and explain why I might be seeing this. I shared the issue with @matlabbe the maintainer of RTAB_MAP and he implemented cpu_tsdf as a post processing step. he clearly understood the detail needed to integrate the libary directly into the application.

I think we can close, since the issue has been addressed and there is a working solution now... but if you want to see the organized pointcloud that rtabmap produces, can certainly share a dataset with you.

let me know, and thanks for following through!

[matlabbe]

The problem was that RTAB-Map exported organized clouds, but in a different frame (e.g., map frame). CPU-TSDF seems to assume that organized clouds are in camera frame (which makes sense). I added new output frame option to RTAB-Map when exporting not assembled organized clouds so that they can be exported in camera frame (z forward, x right, y down). The Export poses action has been also updated to select in which frame we want the poses (here in camera frame). So if there are similar or future implementations similar to cpu-tsdf, we could get the data right!

cheers,
Mathieu

[eric-schleicher]

Thanks so much @matlabbe!