About the centerpillar encoder part

[tianweiy]

About the centerpillar encoder part

[tianweiy]

it should be a 10 dim vector. (xyz, intensity, timestamp, 3D pillar center offset, 2D grid offset). Let me check this

[tianweiy]

What other changes (if any) did you make to your code? I can run the demo successfully. And the lidar has shape [XX, 10].

One possibility is that you are running this in the same environment as the original Det3D repository (same conda environment or Det3D is in the bashrc). If this is the case, can you create a new environment for centerpoint and run the demo again? The original pillar encoder in Det3D has some bug that it will only create a 9 dim vector.

[muzi2045]

if I want to inference the network on my own data bag, just set the timestamp dim to zero will be ok?
"time_lag": curr_sd_rec["timestamp"] * 0
time_lag = ref_time - 1e-6 * curr_sd_rec["timestamp"]

by the way, why only create a 9 dim vector is a "bug"?
In my opinion, adding the timestampe to points dim just because nuscenes dataset annotated the one key frame
and next 9 frame data has no correspoinding annotations, so maybe adding time info will make network more focus on current frame info if the merged scene has too many dynamic obstacles.
but not all datasets are same as nuscenes, using 9-dim vector may be more general practice.
just personal opinion.
@tianweiy

[tianweiy]

In my opinion, adding the timestampe to points dim just because nuscenes dataset annotated the one key frame
and next 9 frame data has no correspoinding annotations, so maybe adding time info will make network more focus on current frame info if the merged scene has too many dynamic obstacles.
but not all datasets are same as nuscenes, using 9-dim vector may be more general practice.
just personal opinion.

I mean the original Det3D pillar encoder is hardcoded to only take 4 dim vectors for lidar points. So even if you specify to use 5 dimension lidar point, it will still ignore the timestamp, which is a bug. There are also other bugs in the pillar implementation in Det3D, though my latest pull request has fixed all of them.

Another thing about multisweeps is for velocity estimation, which is only reasonable with multiple sweep of lidar points.

9-dim / 10-dim is just personal choice for dataset. You only need to change the loading.py and the input_dim field in the config.

if I want to inference the network on my own data bag, just set the timestamp dim to zero will be ok?

I guess so.

[muzi2045]

got it, thanks a lot.