ROS inference file "multi_sweep_inference.py" bad callback: <function rslidar_callback at 0x7fdebadcf2f0> IndexError: invalid index to scalar variable

[tianweiy]

ROS inference file "multi_sweep_inference.py" bad callback: IndexError: invalid index to scalar variable

from centerpoint.

[tianweiy]

glad to hear. Pillar is our fastest / most inaccurate model so you can try other stronger model like https://github.com/tianweiy/CenterPoint/blob/master/configs/centerpoint/nusc_centerpoint_voxelnet_01voxel.py You also want to tune this function to get desired number of output

CenterPoint/tools/multi_sweep_inference.py

Line 68 in caa2f76

def remove_low_score_nu(image_anno, thresh):

[tianweiy]

not sure about rivz details. Tracking is just detection + ID so I guess it is possible.

[tianweiy]

@iamsiddhantsahu at

CenterPoint/tools/multi_sweep_inference.py

Line 187 in caa2f76

shape=grid_size

Change to shape = [grid_size]

[iamsiddhantsahu]

@tianweiy Yes, I have already tried that, but still getting this error

[ERROR] [1595862420.362335]: bad callback: <function rslidar_callback at 0x7f56679602f0>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/home/siddhantsahu/centerpoint_p3_ws/src/centerpoint_ros_node/src/tools/multi_sweep_inference.py", line 339, in rslidar_callback
    scores, dt_box_lidar, types = proc_1.run()
  File "/home/siddhantsahu/centerpoint_p3_ws/src/centerpoint_ros_node/src/tools/multi_sweep_inference.py", line 192, in run
    outputs = self.net(self.inputs)[0] #this is where the error is
  File "/home/siddhantsahu/miniconda3/envs/centerpoint/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/models/detectors/point_pillars.py", line 52, in forward
    return self.bbox_head.loss(example, preds)
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/models/bbox_heads/mg_head.py", line 1294, in loss
    hm_loss = self.crit(preds_dict['hm'], example['hm'][task_id])
KeyError: 'hm'

[tianweiy]

change

CenterPoint/tools/multi_sweep_inference.py

Line 192 in caa2f76

outputs = self.net(self.inputs)[0]

to

outputs = self.net(self.inputs, return_loss=False)[0] 

[iamsiddhantsahu]

Still

[ERROR] [1595862876.321550]: bad callback: <function rslidar_callback at 0x7f56497ac2f0>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/home/siddhantsahu/centerpoint_p3_ws/src/centerpoint_ros_node/src/tools/multi_sweep_inference.py", line 339, in rslidar_callback
    scores, dt_box_lidar, types = proc_1.run()
  File "/home/siddhantsahu/centerpoint_p3_ws/src/centerpoint_ros_node/src/tools/multi_sweep_inference.py", line 192, in run
    outputs = self.net(self.inputs, return_loss=False)[0] #this is where the error is
  File "/home/siddhantsahu/miniconda3/envs/centerpoint/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/models/detectors/point_pillars.py", line 54, in forward
    return self.bbox_head.predict(example, preds, self.test_cfg)
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/models/bbox_heads/mg_head.py", line 1459, in predict
    task_id=task_id
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/core/utils/center_utils.py", line 392, in ddd_decode
    keep = _circle_nms(boxes, min_radius=cfg.min_radius[task_id], post_max_size=cfg.post_max_size)
  File "/home/siddhantsahu/Desktop/CenterPoint/det3d/core/utils/center_utils.py", line 274, in _circle_nms
    keep = np.array(circle_nms(boxes.cpu().numpy(), thresh=min_radius))[:post_max_size]
RuntimeError: Can't call numpy() on Variable that requires grad. Use var.detach().numpy() instead.

[tianweiy]

        with torch.no_grad():
            outputs = self.net(self.inputs, return_loss=False)[0]

[tianweiy]

Are there still any other errors?

[iamsiddhantsahu]

@tianweiy Thanks a lot! Yes, there are no errors now and the predictions also seems to match with the annotation ground truth. But there appears to be more number of unnecessary predicted boxes.

The transparent boxes are ground truth and solid ones are predicted. I am using the centerpoint_pillar_512 trained model.

[iamsiddhantsahu]

Sure, will try with the best trained models. Fine tuning the thresh definitely helps.
The model which I tried nusc_centerpoint_pp_02voxel_circle_nms_demo is it only detection or detection + tracking as well? Can I try to use the centerpoint_voxel_1440_dcn_flip(Rotated NMS) tracking model and other tracking pre-trained models well?

[tianweiy]

only detection. Tracking script is here. You can use all pre-trained models but currently, the two ros scripts don't support the flip data augmentation.

[iamsiddhantsahu]

In addition to detection can tracking be visualized in ROS rivz as well?

[muzi2045]

check the id info in output tensor and add it in visual box.

[iamsiddhantsahu]

@muzi2045 Thanks for suggesting. Should a new ROS topic needs to created and the id data needs to be published there or it can be published to an existing ROS topic? And how to visualize the respective id over their bounding boxes?

[YoushaaMurhij]

check the id info in output tensor and add it in visual box.

The output tensor contains box3d_lidar, scores and label_preds. There is no id info. Can you provide more information, please?

[tianweiy]

@YoushaaMurhij Yeah, no id info in the tensor. You need to follow the tools/tracking/pub_test.py to also generate id info in the ros inference file.

[xavidzo]

@tianweiy Thanks a lot! Yes, there are no errors now and the predictions also seems to match with the annotation ground truth. But there appears to be more number of unnecessary predicted boxes.

The transparent boxes are ground truth and solid ones are predicted. I am using the centerpoint_pillar_512 trained model.

HI @iamsiddhantsahu, one queston: I tried to reproduce the visualization of this screenshot in rviz, but only the predicted bounding boxes (i.e. only solid boxes) are shown, which come from the / pp_boxes topic. How did you manage to display also the ground truth boxes (transparent ones) at the same time? I didn't see any other topic from the rosbag file publishing this information.
I would appreciate your help, thanks.