This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation"
The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such a problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing scalability and performance. In this paper, we propose an instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. Besides, we propose the region-guided regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. Our method is so concise and efficient that it is easy to be generalized to other unsupervised domain adaptation methods. Experiments on 'GTA5 to Cityscapes' and 'SYNTHIA to Cityscapes' demonstrate the superior performance of our approach compared with the state-of-the-art methods.
source | target | device | GPU memory | mIoU-19 | mIoU-16 | mIoU-13 | model |
---|---|---|---|---|---|---|---|
GTA5 | Cityscapes | Tesla V100-32GB | 18.5 GB | 51.88 | - | - | download |
GTA5 | Cityscapes | Tesla T4 | 6.3 GB | 51.20 | - | - | download |
SYNTHIA | Cityscapes | Tesla V100-32GB | 18.5 GB | - | 51.54 | 57.81 | download |
SYNTHIA | Cityscapes | Tesla T4 | 6.3 GB | - | - | - |
- Pytorch >= 1.0
- Python >= 3.6
- cuda >= 9.0
Install python packages
$ pip install -r requirements.txt
apex
: Tools for easy mixed precision and distributed training in Pytorch
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
Please download the datasets from these links:
Dataset directory should have this structure:
${ROOT_DIR}/data/GTA5/
${ROOT_DIR}/data/GTA5/images
${ROOT_DIR}/data/GTA5/labels
${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES
${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES/RGB
${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES/GT
${ROOT_DIR}/data/cityscapes
${ROOT_DIR}/data/cityscapes/leftImg8bit
${ROOT_DIR}/data/cityscapes/gtFine
We provide pre-trained models. We recommend that you download them and put them in pretrained_models/
, which will save a lot of time for training and ensure consistent results.
V100 models
- GTA5 to Cityscapes: G_gtav_at_warmup_v100.pth and M_gtav_at_warmup_v100.pth
- SYNTHIA to Cityscapes: G_syn_at_warmup_v100.pth and M_syn_at_warmup_v100.pth
T4 models
- GTA5 to Cityscapes: G_gtav_at_warmup_t4.pth and M_gtav_at_warmup_t4.pth
- SYNTHIA to Cityscapes: coming soon
(Optional) Of course, if you have plenty of time, you can skip this step and start training from scratch. We also provide these scripts.
Our original experiments are all carried out on Tesla-V100, and there will be a large number of GPU memory usage (batch_size=8
). For low GPU memory devices, we also trained on Tesla-T4 to ensure that most people can reproduce the results (batch_size=2
).
Start self-training (download the pre-trained models first)
cd code
# GTA5 to Cityscapes (V100)
sh ../scripts/self_training_only/run_gtav2cityscapes_self_traing_only_v100.sh
# GTA5 to Cityscapes (T4)
sh ../scripts/self_training_only/run_gtav2cityscapes_self_traing_only_t4.sh
# SYNTHIA to Cityscapes (V100)
sh ../scripts/self_training_only/run_syn2cityscapes_self_traing_only_v100.sh
# SYNTHIA to Cityscapes (T4)
sh ../scripts/self_training_only/run_syn2cityscapes_self_traing_only_t4.sh
(Optional) Training from scratch
cd code
# GTA5 to Cityscapes (V100)
sh ../scripts/from_scratch/run_gtav2cityscapes_self_traing_v100.sh
# GTA5 to Cityscapes (T4)
sh ../scripts/from_scratch/run_gtav2cityscapes_self_traing_t4.sh
# SYNTHIA to Cityscapes (V100)
sh ../scripts/from_scratch/run_syn2cityscapes_self_traing_v100.sh
# SYNTHIA to Cityscapes (T4)
sh ../scripts/from_scratch/run_syn2cityscapes_self_traing_t4.sh
cd code
python eval.py --config_file <path_to_config_file> --resume_from <path_to_*.pth_file>
Support multi-scale testing and flip testing.
# Modify the following parameters in the config file
TEST:
RESIZE_SIZE: [[1024, 512], [1280, 640], [1536, 768], [1800, 900], [2048, 1024]]
USE_FLIP: False
Please cite this paper in your publications if it helps your research:
@article{mei2020instance,
title={Instance Adaptive Self-Training for Unsupervised Domain Adaptation},
author={Mei, Ke and Zhu, Chuang and Zou, Jiaqi and Zhang, Shanghang},
journal={arXiv preprint arXiv:2008.12197},
year={2020}
}
Ke Mei
- email: [email protected]
- wechat: meikekekeke
If you have any questions, you can contact me directly.