How to reproduce reported performance

[willyd]

How to reproduce reported performance

[chreisinger]

Okay, I use Pytorch 1.3, the newest version of numpy and an unchanged version of spconv. By the way, I use a single 1080Ti.

[sshaoshuai]

The provided pretrained models could achieve exactly the same reported performance. Have you modified the codes? I think you should make true to achieve the same reported performance with the given model firstly.

[chreisinger]

Hi, I have reproduced similar results as reported by training the network on my own (SECOND, Pointpillars) with the proposed commands (small fix for pointpillars necessary see #10 ).

[willyd]

Thanks for the quick reply. I did not modify the code except for a small change in spconv to make it work with pytorch 1.4. I only tried the default config for pointpillars, but did not get the error mentionned by @chreisinger. I guess I could try with 1.1 or 1.3.

@sshaoshuai @chreisinger What pytorch, numpy versions are you using?

[willyd]

Forgot to mention I only have 4 gpus so I changed the batch size to 28 per GPU instead of 32. I guess should have used 16 as recommended instead.

But that would not explain the poor results obtained with the pretained model.

[willyd]

The issue was with a corrupted KITTI dataset I got from someone else... Sorry for the noise and thanks for the support. I can reproduce the exact same numbers with the PointPillars pretrained model with pytorch 1.4.