f-pointnet：a big gap between test in my environment and the your screenshot,can you help me find the problem?

[horizon-research]

f-pointnet：a big gap between test in my environment and the your screenshot,can you help me find the problem?

[YuFengUofR]

Hi, you didn't do anything wrong, we didn't uploaded the original model weights to the Github, because we refactored and pushed the code after we published the paper, the original weights cannot be used for the refactored code, we just uploaded the weights that make the network runnable, maybe longer training could help on accuracy.

[anjingde]

Thank you very much! I will try train longer . Great job!

[anjingde]

How many rounds do you usually train？ I try to train 300 epoch ,but I can't close the gap。 can you help me find the problem?

10 epoch eval
Going to eval ground for class: car || your result
save detection_results_v2/plot/car_detection_ground.txt
car_detection_ground AP: 81.561928 72.300224 63.747066 || 86.406395 81.9703 74.644371
Going to eval ground for class: pedestrian
save detection_results_v2/plot/pedestrian_detection_ground.txt
pedestrian_detection_ground AP: 56.336613 48.243210 44.063805 | | 72.302208 66.122513 59.363037
Going to eval ground for class: cyclist
save detection_results_v2/plot/cyclist_detection_ground.txt
cyclist_detection_ground AP: 56.276375 39.054710 36.133801 || 84.09743 64.391205 60.237617
Finished Birdeye eval.

151 epoch
Going to eval ground for class: car
save detection_results_v2/plot/car_detection_ground.txt
car_detection_ground AP: 84.826126 75.892754 72.312393
Going to eval ground for class: pedestrian
save detection_results_v2/plot/pedestrian_detection_ground.txt
pedestrian_detection_ground AP: 54.360111 47.633389 43.841793
Going to eval ground for class: cyclist
save detection_results_v2/plot/cyclist_detection_ground.txt
cyclist_detection_ground AP: 58.887108 42.317303 39.416611

200
save detection_results_v2/plot/car_detection_ground.txt
car_detection_ground AP: 85.156235 75.999329 72.396149 | 86.406395 81.9703 74.644371
Going to eval ground for class: pedestrian
save detection_results_v2/plot/pedestrian_detection_ground.txt
pedestrian_detection_ground AP: 55.607517 48.458321 44.497604 | 72.302208 66.122513 59.363037
Going to eval ground for class: cyclist
save detection_results_v2/plot/cyclist_detection_ground.txt
cyclist_detection_ground AP: 57.910442 42.495819 39.184010 | 84.09743 64.391205 60.237617

222
save detection_results_v2/plot/car_detection_ground.txt
car_detection_ground AP: 84.650726 75.870003 72.364899
Going to eval ground for class: pedestrian
save detection_results_v2/plot/pedestrian_detection_ground.txt
pedestrian_detection_ground AP: 55.311619 47.841358 43.998032
Going to eval ground for class: cyclist
save detection_results_v2/plot/cyclist_detection_ground.txt
cyclist_detection_ground AP: 59.439980 43.064941 40.021294

300

save detection_results_v2/plot/car_detection_ground.txt
car_detection_ground AP: 83.637230 75.204834 66.471741
Going to eval ground for class: pedestrian
save detection_results_v2/plot/pedestrian_detection_ground.txt
pedestrian_detection_ground AP: 55.506302 48.111755 44.167988
Going to eval ground for class: cyclist
save detection_results_v2/plot/cyclist_detection_ground.txt
cyclist_detection_ground AP: 53.105473 37.960785 34.334435
Finished Birdeye eval.

environment:
a 1080ti
ubuntu16.04
python 2.7
tensorflow1.9.0 cuda9.0 cudnn 7.6.5 numpy 1.16.6.

[anjingde]

hello, I really need your help?

[YuFengUofR]

I think there are several possibilities that could lead to low accuracy. I could upload the previous code soon, if you wait for a second, I will clean up code and upload the code soon.

[anjingde]

Thank you very much !

[anjingde]

hello, I also have the same problem. the runtime is 0.011-0.014s (11 ms -14 ms )in 1080 ti (test_runtime.py), but in the paper of frustum pointnet the runtime is 170 ms . In 3dssd the runtime is (sa (4)layer fp (4) layer refinement module(3)) : 40ms 14 ms 35 ms . I think the result is not logical. So is there a problem with this part of the code? or my understanding is wrong? Thank you!

[anjingde]

I know the code don't include the inference time of 2d detection.

[YuFengUofR]

It depends on what gpu you used. If you used something like Nvidia 1080, there is no speedup, because the network is quite light-compute, tthe GPU resource is good enough for both baseline and our new method. If you want to see the difference, change the code to make the batch_size to be a big number, like 16, you will see the speedup. Our method targets on mobile devices like TX2. If you test on those devices, you will see the significant speedup.

[anjingde]

Thank you,I Have saw this answer, but I don't konw why the runtime is only 11-15ms (model_baseline or model) , It is not logical.

[anjingde]

When I eval the val (the baseline model ) ,I meet the problem. How to deal with this problem?
Other evaluation models worked normally

car_detection_ground AP: 83.765503 74.609375 65.553574
Going to eval ground for class: pedestrian
Segmentation fault (core dumped)

[YuFengUofR]

yeah, I came across it a couple of times. It is from the original repo: https://github.com/charlesq34/frustum-pointnets