zhechen/plard

Progressive LiDAR Adaptation for Road Detection

plard's Issues

Bad test result using pretrained model

Hi,

After running the test with the pre-trained model, the output results are stored in /output/results.
By using the evaluation code from devkit_road download from KITTI website, the following is the result:
Starting evaluation ...
Available categories are: ['um_lane', 'um_road', 'umm_road', 'uu_road']
Execute evaluation for category um_lane ...
Searching for submitted files with prefix: um_lane_
Computing evaluation scores...
MaxF: 86.93
AvgPrec: 90.74
PRE_wp: 86.91
REC_wp: 86.95
FPR_wp: 1.17
FNR_wp: 13.05
Finished evaluating category: um_lane
Execute evaluation for category um_road ...
Searching for submitted files with prefix: um_road_
Computing evaluation scores...
MaxF: 89.62
AvgPrec: 93.21
PRE_wp: 89.32
REC_wp: 89.93
FPR_wp: 2.12
FNR_wp: 10.07
Finished evaluating category: um_road
Execute evaluation for category umm_road ...
Searching for submitted files with prefix: umm_road_
Computing evaluation scores...
MaxF: 83.00
AvgPrec: 89.42
PRE_wp: 77.65
REC_wp: 89.14
FPR_wp: 8.03
FNR_wp: 10.86
Finished evaluating category: umm_road
Execute evaluation for category uu_road ...
Searching for submitted files with prefix: uu_road_
Computing evaluation scores...
MaxF: 81.50
AvgPrec: 87.35
PRE_wp: 79.03
REC_wp: 84.13
FPR_wp: 3.52
FNR_wp: 15.87
Finished evaluating category: uu_road
Successfully finished evaluation for 4 categories: ['um_lane', 'um_road', 'umm_road', 'uu_road']

Apparently, the result is far from what you posted in your paper.
Would you please help me figure out why?
Btw, a lot of bugs are found in the evaluation code, and I have to correct them in order to run.

Pretrained Model for Training

Hi,

Thank you very much for your code. Could I know which model is used as the pre-trained model in your training?

The PSP model on CitySpaces or the resnet101 pre-trained on ImageNet.

Many thanks

RuntimeError: CUDA out of memory

Hi,
Now, I can replicate the result of the test using GPUs.
However, when I train the model with the following command:
python train.py --arch plard --dataset kitti_road --no-visdom --pretrained ./plard_kitti_road.pth
I get the error:

below is the error info:

Found 289 train images

Params: l_rate 0.000050, l_rate_decay: 0.10, l_rate_step: 1, batch_size: 4, mom: 0.99, wd: 0.000500

Using custom loss

/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/functional.py:2416: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.

warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/_reduction.py:13: UserWarning: reduction='elementwise_mean' is deprecated, please use reduction='mean' instead.
warnings.warn("reduction='elementwise_mean' is deprecated, please use reduction='mean' instead.")
Traceback (most recent call last):
File "train.py", line 185, in
train(args, logger)
File "train.py", line 101, in train
loss = model([images, lidars, labels])
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 85, in parallel_apply
output.reraise()
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker
output = module(*input, **kwargs)
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/u4067/Documents/PLARD/PLARD_intact/ptsemseg/models/plard.py", line 358, in forward
loss.backward()
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/autograd/init.py", line 99, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: CUDA out of memory. Tried to allocate 120.00 MiB (GPU 0; 7.79 GiB total capacity; 6.37 GiB already allocated; 116.00 MiB free; 6.53 GiB reserved in total by PyTorch)

Could you please help with this error?
My env:
ubuntu: 16.4
python: 3.7.0
pytorch: 1.4.0
torchfile: 0.1.0
torchversion: 0.2.0
GPU: Geforce RTX 2070
cuda: 10.0
And changing the batch size doesn't help.

How to get the summary of the model?

I tried to get the summary of the model. For That we needed input size.

On investigation it was found out that the shape of inputs are different for RGB and LiDAR Model.

RGB Model has -> (1,3,384,1280)
Lidar Mode has -> (1,1,384,1280)

Pleas confirm what should be the input shape for the model so that Model summary can be shown.

Thanks

thank you

Which lidar sensor you have used very good code thank you

projection of lidar data using ADI matlab code

I am just trying to use my own calibration data using your ADI code in matlab. but it does not work properly. May I ask how did you re-generate calibration format of Kitti? how did you calibrate your camera and lidar? My calib info are coming from autoware lidar-camera calibration and transforming it to kitti format has not been successful.

NF2CNN?

Hello, where can i find the relevant information of NF2CNN network.

LOSS stacks at round 0.45

Original code without changing anything.
But LOSS stacks at around 0.45 after 20 epoches.

Could you help me?
Have tried LR from 1e-4 to 1e-5

Got very bad results using Test.py and a trained model with 0.45 loss.

loss 不下降

感谢你的开源，最近在试着训练你的网络，发现loss并不下降，从1左右迅速下降到0.5之后（不到一个epoch），就不再下降，而且训练出来的权重，测试结果完全没法看，铁别差。环境和你的一样，请问这大概是什么原因呢？谢谢！

Cuda error

Hi,

When I run the code, I got:
RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS.
Would you please help find out what the problem is?
My env:
ubuntu: 16.4
python: 3.7.0
pytorch: 0.4.1
torchfile: 0.1.0
torchversion: 0.2.0
GPU: Geforce RTX 2070
cuda: 10.0

below is the error info:

Building plard Found 290 test images processing 0-th image Exception ignored in: Traceback (most recent call last): File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 399, in __del__ self._shutdown_workers() File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 378, in _shutdown_workers self.worker_result_queue.get() File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/multiprocessing/queues.py", line 354, in get return _ForkingPickler.loads(res) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 151, in rebuild_storage_fd fd = df.detach() File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/multiprocessing/resource_sharer.py", line 58, in detach return reduction.recv_handle(conn) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/multiprocessing/reduction.py", line 185, in recv_handle return recvfds(s, 1)[0] File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/multiprocessing/reduction.py", line 153, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) ConnectionResetError: [Errno 104] Connection reset by peer processing 0-th image Traceback (most recent call last): File "test.py", line 82, in test(args) File "test.py", line 55, in test outputs = model([tr_image, tr_lidar]) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__ result = self.forward(*input, **kwargs) File "/home/u4067/Documents/PLARD/PLARD_intact/ptsemseg/models/plard.py", line 242, in forward x = self.convbnrelu1_1(x) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__ result = self.forward(*input, **kwargs) File "/home/u4067/Documents/PLARD/PLARD_intact/ptsemseg/models/utils.py", line 92, in forward outputs = self.cbr_unit(inputs) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__ result = self.forward(*input, **kwargs) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/container.py", line 91, in forward input = module(input) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__ result = self.forward(*input, **kwargs) File "/home/u4067/anaconda3/envs/py_plard/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 301, in forward self.padding, self.dilation, self.groups) RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS

trained model

Thank you for your open source!Now i need use your trained model to test my image,but your trained model in here can't download.So please tell me what can i do.Thanks again.

can't find load_state_dict_without_classification

Thank you for your work，when i train the model,I can't find the definition of the fllowing functions in train.py line 44:model.load_state_dict_without_classification(checkpoint['model_state']). and line 98: model.module.freeze_bn().Is there any missing document？Looking forward to your reply.

Hello, I am learning your work,but I can not open the url of ADI download link,could you help me?

PLARD training

Hi,

Thank you very much for your code.

I use the pspnet101_cityscapes pre-trained model to train kitti, but I cannot achieve the results of the paper. Can you provide suggestions?

dataset: kitti training 289 sheets
model: PLARD (no modification)
batch_size: 8
Learning Rate: 1e-4 -> 1e-6 (SGD)
epoch: 80

Many thanks

ADT

thanks a lot for this work.
Could you also share the code of Data space adaptation step?