[Problem] very low recall_rcnn_* on my custum dataset

[xzswater]

[Problem] very low recall_rcnn_* on my custum dataset

[sshaoshuai]

I could not help much with the limited information, maybe you should first try a simpler one-stage method like SECOND or PointPillar to achieve normal results.

[xzswater]

Thank you so much. Could you provide some suggestions on how to check the quality of rpn rois ?
Does low recall_roi_* relative to voxels and spconv ?

[sshaoshuai]

There could be many problems which causes the low performance. Some suggestions are as follows:
(1) Draw the output of your model to see if it is as expected.
(2) Check your labels to see if it is same with KITTI dataset. Especially double check that the input of gt_boxes to the network should be (x, y, z, w, l, h, ry) in the KITTI LiDAR coordinate, here the (x, y, z) is the bottom center of the object.

[xzswater]

Thank you very much !

1. I will visualize points, model input and output soon.
2. Ours coordinates different from KITTI, so i modified lots of codes (include spconv) to transfer coords. But, kept voxel generators and model architecture codes as originally.

[asadnorouzi]

@sshaoshuai I have retrained PartA^2 with Pseudo-LiDAR data instead of the original Velodyne data. I am getting very bad results. My initial assumption is that your proposed framework relies on actual LiDAR data (eg. Velodyne) and doesn't perform well with peudo-lidar data. I tested my pseudo-lidar data with SECOND and the results were reasonable. I'd appreciate to have your opinion about this.

[sshaoshuai]

@asadnorouzi Sorry I never tested it with the Pseudo-LiDAR. Since the Pseudo-LiDAR is denser and has more noises, the point-based refinement of RCNN stage may not perform well enough. But it is still strange for the very bad results since the RCNN stage should get better results after refine the RPN proposals. I think it may need more time to tune on the Pseudo-LiDAR.

[asadnorouzi]

@sshaoshuai Thank you for your reply. I was thinking to let it train longer than 80 epochs. Maybe 160 or even 240 epochs. What do you think?

[sshaoshuai]

I don't think more epochs will work well if 80 epochs doesn't work at all, but you could have a try.

[neverrop]

@xzswater how did u make your own dataset, can you share something for this? thanks.

[clytze0216]

@asadnorouzi抱歉，我从未使用伪激光雷达对其进行过测试。由于 Pseudo-LiDAR 更密集且噪声更多，因此 RCNN 阶段的基于点的细化可能表现得不够好。但是非常糟糕的结果仍然很奇怪，因为在改进 RPN 建议之后，RCNN 阶段应该会得到更好的结果。我认为可能需要更多时间来调整 Pseudo-LiDAR。

I custom my own data to train the second model. But I get very low recall ,too. I want to ask how do you solve this problem? Looking forward to your reply！

[asadnorouzi]

@clytze0216 What is your custom data? My data was based on pseudo-lidar points. These points were generated by depth map estimator and then fed into the SECOND model. The accuracy will be much lower than lidar-based points, but this low accuracy is expected for pseudo-lidar points.

[clytze0216]

@asadnorouzi
My custom data is the point cloud information collected and annotated by myself.
I only have 3D XYZ and HWL information and YAW.I've commented out all the code for the 2D coordinate information and any other data that I don't have.I found that the test. PKL file I generated during the data processing phase was very small, around 40KB, and the recall in the eval result was very low, around 20.
Could you give me some advice?This is very important to me. I am looking forward to your reply. Thank you!