linjiefu-u/dynamicrouting

Learning Dynamic Routing for Semantic Segmentation

DynamicRouting

This project provides an implementation for "Learning Dynamic Routing for Semantic Segmentation" (CVPR2020 Oral) on PyTorch. For the reason that experiments in the paper were conducted using internal framework, this project reimplements them on dl_lib and reports detailed comparisons below. Some parts of code in dl_lib are based on detectron2.

Requirement

• Python >= 3.6
  • python3 --version
• PyTorch >= 1.3
  • pip3 install torch torchvision
• OpenCV
  • pip3 install opencv-python
• GCC >= 4.9
  • gcc --version

Installation

Make sure that your get at least one gpu when compiling. Run:

• git clone https://github.com/yanwei-li/DynamicRouting.git
• cd DynamicRouting
• sudo python3 setup.py build develop

Usage

Dataset

We use Cityscapes dataset for training and validation. Please refer to datasets/README.md or dataset structure in detectron2 for more details.

• Cityscapes Download

Pretrained Model

We give ImageNet pretained models:

• Layer16-Fix GoogleDrive
• Layer33-Fix GoogleDrive

Training

For example, if you want to train Dynamic Network with Layer16 backbone:

• Train from scratch

  cd playground/Dynamic/Seg.Layer16
  dl_train --num-gpus 4

• Use ImageNet pretrain

  cd playground/Dynamic/Seg.Layer16.ImageNet
  dl_train --num-gpus 4 MODEL.WEIGHTS /path/to/your/save_dir/ckpt.pth

NOTE: Please set FIX_SIZE_FOR_FLOPS to [768,768] and [1024,2048] for training and evaluation, respectively.

Evaluation

You can evaluate the trained or downloaded model:

• Evaluate the trained model

  dl_test --num-gpus 8

• Evaluate the downloaded model:

  dl_test --num-gpus 8 MODEL.WEIGHTS /path/to/your/save_dir/ckpt.pth 

NOTE: If your machine does not support such setting, please change settings in config.py to a suitable value.

Performance

Cityscapes val set

Without ImageNet Pretrain:

Methods	Backbone	Iter/K	mIoU (paper)	GFLOPs (paper)	mIoU (ours)	GFLOPs (ours)	Model
Dynamic-A	Layer16	186	72.8	44.9	73.9	52.5	GoogleDrive
Dynamic-B	Layer16	186	73.8	58.7	74.3	58.9	GoogleDrive
Dynamic-C	Layer16	186	74.6	66.6	74.8	59.8	GoogleDrive
Dynamic-Raw	Layer16	186	76.1	119.5	76.7	114.9	GoogleDrive
Dynamic-Raw	Layer16	558	78.3	113.3	78.1	114.2	GoogleDrive

With ImageNet Pretrain:

Methods	Backbone	Iter/K	mIoU (paper)	GFLOPs (paper)	mIoU (ours)	GFLOPs (ours)	Model
Dynamic-Raw	Layer16	186	78.6	119.4	78.8	117.8	GoogleDrive
Dynamic-Raw	Layer33	186	79.2	242.3	79.4	243.1	GoogleDrive

To do

• Faster inference speed
• Support more vision tasks
  • Object detection
  • Instance segmentation
  • Panoptic segmentation

Acknowledgement

• Detectron2
• DARTS

Citation

Consider cite the Dynamic Routing in your publications if it helps your research.

@inproceedings{li2020learning,
    title = {Learning Dynamic Routing for Semantic Segmentation},
    author = {Yanwei Li, Lin Song, Yukang Chen, Zeming Li, Xiangyu Zhang, Xingang Wang, Jian Sun},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    year = {2020}
}

Consider cite this project in your publications if it helps your research.

@misc{DynamicRouting,
    author = {Yanwei Li},
    title = {DynamicRouting},
    howpublished = {\url{https://github.com/yanwei-li/DynamicRouting}},
    year ={2020}
}