camera/rgb... sensor_msgs/Image Messages don't render in the image view

[cruise-automation]

camera/rgb... sensor_msgs/Image Messages don't render in the image view

[brianc]

@palafranchise We have manually created image decoders for the images encodings we use inside our bags at Cruise. It's possible you are encoding images in formats we don't have support for yet. What encodings are you using? You should be able to get this from the raw message inspector for an image message on the "encoding" field.

@vidaaudrey we could add support for bz2 compression in the open-source version at some point. Rosbag.js supports this no problemo; it's just a matter of supplying it to the bag data provider's rosbag options.

[palafranchise]

Update: I tried running locally but didn't get any additional errors or change in behavior.

[vidaaudrey]

I saw that you just posted about the wasm error, is it gone? Would be helpful if you could share a sample bag with us so we can check the issue in details.

[palafranchise]

I thought the wasm error was unrelated... pasted below for reference. I'll see if I can find a way to upload the bag file I'm testing with.

Uncaught (in promise) abort("abort(\"on the web, we need the wasm binary to be preloaded and set on Module['wasmBinary']. emcc.py will do that for you when generating HTML (but not JS)\"). Build with -s ASSERTIONS=1 for more info."). Build with -s ASSERTIONS=1 for more info.

[palafranchise]

I've been using this bag file which has both "/camera/depth/image" and "/camera/rgb/image_color" topics with Image messages.

https://vision.in.tum.de/rgbd/dataset/freiburg1/rgbd_dataset_freiburg1_xyz.bag

Note: I had to decompress it.

Here's the rosbag info output once I decompressed it:

rosbag info rgbTest.bag 
path:        rgbTest.bag
version:     2.0
duration:    30.4s
start:       May 10 2011 12:38:18.38 (1305031098.38)
end:         May 10 2011 12:38:48.81 (1305031128.81)
size:        1.6 GB
messages:    25626
compression: none [1598/1598 chunks]
types:       sensor_msgs/CameraInfo         [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image              [060021388200f6f0f447d0fcd9c64743]
             sensor_msgs/Imu                [6a62c6daae103f4ff57a132d6f95cec2]
             tf/tfMessage                   [94810edda583a504dfda3829e70d7eec]
             visualization_msgs/MarkerArray [f10fe193d6fac1bf68fad5d31da421a7]
topics:      /camera/depth/camera_info     798 msgs    : sensor_msgs/CameraInfo        
             /camera/depth/image           798 msgs    : sensor_msgs/Image             
             /camera/rgb/camera_info       798 msgs    : sensor_msgs/CameraInfo        
             /camera/rgb/image_color       798 msgs    : sensor_msgs/Image             
             /cortex_marker_array         3034 msgs    : visualization_msgs/MarkerArray
             /imu                        15158 msgs    : sensor_msgs/Imu               
             /tf                          4242 msgs    : tf/tfMessage

I found this bag file here: https://vision.in.tum.de/rgbd/dataset/

[palafranchise]

Do you have a working bag file you can share?

[vidaaudrey]

We are working on adding some sample bags. It'll be available soon.

Looks like the bag is using type bz2 compression which we don't yet support. It's hitting the path here:
https://github.com/cruise-automation/rosbag.js/blob/46365742845f343f1a7e17e2145b8e23293dbbcf/src/BagReader.js#L254

Currently the rosbag.js library only support lz4 compression. Might be able to get around by converting. @brianc any suggestions?

[palafranchise]

Yup, the link I sent is to a compressed bag. But I uncompressed it (unless there's an additional compression I'm missing). See the the rosbag info above. I've tried with both uncompressed and lz4 compressed bags but I get the same behavior: the sensor_msgs/Image messages are there but the image view does not render messages on the /camera/rgb/image_color topic.

[palafranchise]

Ah, I see. The images which are not appearing have encoding values of "rgb8". What encoding format did you use?

[palafranchise]

Or, which encoding formats does the tool support?

Think I found it: https://github.com/cruise-automation/webviz/blob/master/packages/webviz-core/src/panels/ImageView/ImageCanvas.js#L89

Looks like it might work if I switch my encoding to bgr8

[palafranchise]

Yup, switching to bgr8 works. Thanks for your help!