zengyi-qin/weakly-supervised-3d-object-detection

Weakly Supervised 3D Object Detection from Point Clouds (VS3D), ACM MM 2020

Weakly Supervised 3D Object Detection from Point Clouds (VS3D)

Created by Zengyi Qin, Jinglu Wang and Yan Lu. The repository contains an implementation of this ACM MM 2020 Paper. Readers are strongly recommended to create and enter a virtual environment with Python 3.6 before running the code.

Quick Demo with Jupyter Notebook

Clone this repository:

git clone https://github.com/Zengyi-Qin/Weakly-Supervised-3D-Object-Detection.git

Enter the main folder and run installation:

pip install -r requirements.txt

Download the demo data to the main folder and run unzip vs3d_demo.zip. Readers can try out the quick demo with Jupyter Notebook:

cd core
jupyter notebook demo.ipynb

Training

Download the Kitti Object Detection Dataset (image, calib and label) and place them into data/kitti. Download the ground planes and front-view XYZ maps from here and run unzip vs3d_train.zip. Download the pretrained teacher network from here and run unzip vs3d_pretrained.zip. The data folder should be in the following structure:

├── data
│   ├── demo
│   └── kitti
│       └── training
│           ├── calib
│           ├── image_2
│           ├── label_2
│           ├── sphere
│           ├── planes
│           └── velodyne
│       ├── train.txt
│       └── val.txt
│   └── pretrained
│       ├── student
│       └── teacher

The sphere folder contains the front-view XYZ maps converted from velodyne point clouds using the script in ./preprocess/sphere_map.py. After data preparation, readers can train VS3D from scratch by running:

cd core
python main.py --mode train --gpu GPU_ID

The models are saved in ./core/runs/weights during training. Reader can refer to ./core/main.py for other options in training.

Inference

Readers can run the inference on KITTI validation set by running:

cd core
python main.py --mode evaluate --gpu GPU_ID --student_model SAVED_MODEL

Readers can also directly use the pretrained model for inference by passing --student_model ../data/pretrained/student/model_lidar_158000. Predicted 3D bounding boxes are saved in ./output/bbox in KITTI format.

Citation

@article{qin2020vs3d, 
  title={Weakly Supervised 3D Object Detection from Point Clouds}, 
  author={Zengyi Qin and Jinglu Wang and Yan Lu},
  journal={ACM Multimedia},
  year={2020}
}

weakly-supervised-3d-object-detection's Issues

Getting number from the paper

Hi, I followed the steps in the README with the intention of getting the predictions on the kitti validation set but currently I get <10% using the pre-trained weights.
To evaluate I use the kitti object evaluation script with the threshold set to 0.5 for cars and restricting the files read to only the validation set.
Is there any tricks to getting this to work or advice on how you run evaluation on the outputs?
The values in the output files look incorrect compared to the GT.

Similar Paper

I read another paper with almost the same name
Weakly Supervised 3D Object Detection from Lidar Point Cloud
https://arxiv.org/pdf/2007.11901.pdf

Could you explain more about the difference?

About GPU

Can we use a Titan XP GPU to replace the Tesla P40 GPU?

Values of the scores of each class for each object

Hello Zengyi-Qin,
How are you?
Sorry for the inconvenience.
I am studying object detection and I read your paper Weakly Supervised 3D Object Detection from Point Clouds. I managed to use the codes you posted on GitHub. However I would like to know if it is possible to get the score values of each detected object. For example, if I train detection by considering objects classified into three classes, then each detected object will have three score values. In other words, a detected object has a corresponding score value for each training class, that is, class1 = Score1, class2 = Score2 and to class3 = Score3. Actually, I would like to know if it is possible to get the score values of each class for each object?

Best regards,
Gledson Melotti.

help

When I want to train the model, I use the command: "python main.py --mode train --gpu GPU_ID" uses the CPU. If I want to run the code on the GPU, how should I use the command?

Some questions of evaluation.

Hello, I'm very interested in your wso3d project. Did you use AP_11 in your evaluation or AP_40? I reproduced your model and have found AP_11 score is more similar to your result in your paper.