mecarill/2pcnet

2PCNet: Two-Phase Consistency Training for Day-to-Night Unsupervised Domain Adaptive Object Detection

This repo is the official implementation of our paper:
2PCNet: Two-Phase Consistency Training for Day-to-Night Unsupervised Domain Adaptive Object Detection
Mikhail Kennerley, Jian-Gang Wang, Bharadwaj Veeravalli, Robby T. Tan
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper]

Installation

Prerequisites

• Python ≥ 3.6
• PyTorch ≥ 1.5 and torchvision that matches the PyTorch installation.
• Detectron2 == 0.6

Dataset download

1. Download the datasets (BDD100K / SHIFT)

2. Split BDD100K and SHIFT into day and night labels using dataset information. Convert BDD100K and SHIFT labels to coco format. Alternatively, you can download our split (https://www.dropbox.com/scl/fo/258uzp6i0dz17zsj234r6/h?dl=0&rlkey=kb6brfk1oqc1ddsa3ulz8v9ei).

3. Organize the dataset with the following format

2pcnet/
└── datasets/
    └── bdd100k/
        ├── train/ 
            ├── img00001.jpg
            ├──...
        ├── val/ 
            ├── img00003.jpg
            ├──...
        ├── train_day.json
        ├── train_night.json
        ├── val_night.json
    └── shift/
        ├── train/ 
            ├── folder1
            ├──...
        ├── val/ 
            ├── folder1
            ├──...
        ├── train_day.json
        ├── train_night.json
        ├── val_night.json

    

Training

python train_net.py \
      --num-gpus 4 \
      --config configs/faster_rcnn_R50_bdd100k.yaml\
      OUTPUT_DIR output/bdd100k

Resume the training

python train_net.py \
      --resume \
      --num-gpus 4 \
      --config configs/faster_rcnn_R50_bdd100k.yaml MODEL.WEIGHTS <your weight>.pth

Evaluation

python train_net.py \
      --eval-only \
      --config configs/faster_rcnn_R50_bdd100k.yaml \
      MODEL.WEIGHTS <your weight>.pth

Pretrained Weights

Dataset	Model Link
BDD100K	https://www.dropbox.com/s/812l6wdbonabp9k/model_final.pth?dl=0
SHIFT	Coming soon...

Citation

If you use 2PCNet in your research or wish to refer to the results published in our paper, please use the following BibTeX entry:

@inproceedings{kennerley2023tpcnet,
  title={2PCNet: Two-Phase Consistency Training for Day-to-Night Unsupervised Domain Adaptive Object Detection},
  author={Mikhail Kennerley, Jian-Gang Wang, Bharadwaj Veeravalli, Robby T. Tan},
  booktitle={2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2023},
}

Acknowledgements

Code is adapted from Detectron2 and Adaptive Teacher.

2pcnet's Issues

Visualizing Teacher Prediction

How can we visualize teachers' prediction, student's prediction guided by the teacher, and teachers' prediction for consisteny loss to achieve results similar to those in Fig. 3 of the paper? After training, I think we only have one JSON file representing the student's prediction.

Another question is the intuitive difference between teachers' prediction and teachers' prediction for consisteny loss.

Thanks!

The reported performance of mAP50 in this paper is Student's or Teacher's?

Why is the performance gap so wide between teachers and students?
Their structure is the same, so why are Teacher performing better (about 2~3 point)

Visualization of detection results？

May I ask if the author's code is complete? Can you provide a complete code project that includes visualization of detect results.

Hey,thanks for your excellent work, and where is the code for generating the bdd100k_day(or night).json?

Can you provide APcoco(50:95) of 2PCNet on bdd and shift dataset? pls. TY

The resault of AP50

According to the 100000 iterations mentioned in the paper, I used a 3060GPU, adjusted the batch_size of the source and target domains to 2, and adjusted the learning rate to 0.02. The teacher's result obtained was less than 36, which is far from the 46.4 mentioned in the paper. What is the reason for this?

About the train_day.json and train_night.json of BDD100k

Dear authors:
I appreciate your excellent work. And I would like to reproduce your work on my PC. But when I process the bdd100k dataset there are some problems. Specifically, when I downloaded your split in Dropbox, I found that train_day.json and train_night.json are not included in your Dropbox. I would be extremely grateful if you could upload these two jsons.

Hope to receive your response!

Result not as expected

Hello, authors!

I've used your provided code to train the model, but I'm having trouble reproducing the results on BDD datasets. The achieved average precision (AP) is much lower than what was reported in the paper.

I've already requested a pretrained model in #1. Could you please provide the pretrained model or double-check the current code? Your assistance would be greatly appreciated.

This is my result:

This is the result from the paper:

Update:
Even after removing the category 'train' during evaluation, the mAP is still ~3% lower.

Use of two-stage consistency

The paper demonstrates the significant influence of two-stage consistency on mAP. However, in the code, the "CONSISTENCY: True" configuration parameter is not utilized.

Moreover, I am curious about the performance of two-stage consistency when compared to one-stage consistency, which is commonly employed by many methods, as opposed to no consistency at all.

How can I get the result in paper?

I just get this result

But I don't know how to get the result as shown below

About the BDD100K split json.

Dear authors:
I appreciate your excellent work. And I would like to reproduce your work on my PC. But when I process the bdd100k dataset there are some problems. Specifically, when I downloaded your split in Dropbox, I found that train_day.json and train_night.json are not included in your Dropbox. I would be extremely grateful if you could upload these two jsons.

Hope to receive your response!

best wishes!
Apin.

Pretrained Model

Could you release the pretrained model so I can use it directly for inference?

Hi, it's really a good work! I'm very interested in it. Can you share the datasets you have split to day and night?

backbone model

Dear sir,
Thank you for sharing your work！
I have a question about how to modify the backbone model.The backbone used in your code is Faster RCNN, and I want to modify it to Yolov7.
Since I am just getting started with object detection, I am not familiar with modifying the code. I kindly ask you to provide some ideas for modification (backbone part and train part), and I will modify it by myself according to your ideas.
Looking forward to hearing from you！
Thanks in advance！