pw22-sbn-01/leastereo

An end-to-end nas stereo matching network.

LEAStereo

This repository contains the code for our NeurIPS 2020 paper Hierarchical Neural Architecture Searchfor Deep Stereo Matching [NeurIPS 20]

Requirements

Environment

1. Python 3.8.*
2. CUDA 10.0
3. PyTorch
4. TorchVision

Install

Create a virtual environment and activate it.

conda create -n leastereo python=3.8
conda activate leastereo

The code has been tested with PyTorch 1.6 and Cuda 10.2.

conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.2 -c pytorch
conda install matplotlib path.py tqdm
conda install tensorboard tensorboardX
conda install scipy scikit-image opencv

Install Nvidia Apex

Follow the instructions here. Apex is required for mixed precision training. Please do not use pip install apex - this will not install the correct package.

Dataset

To evaluate/train our LEAStereo network, you will need to download the required datasets.

• SceneFlow

• KITTI2015

• KITTI2012

• Middlebury 2014

Change the first column path in file create_link.sh with your actual dataset location. Then run create_link.sh that will create symbolic links to wherever the datasets were downloaded in the datasets folder. For Middlebury 2014 dataset, we perform our network on half resolution images.

├── datasets
    ├── SceneFlow
        ├── camera_data
        ├── disparity
        ├── frames_finalpass
    ├── kitti2012
        ├── testing
        ├── training
    ├── kitti2015
        ├── testing
        ├── training
    ├── MiddEval3
        ├── testH
        ├── trainingH

Prediction

You can evaluate a trained model using prediction.sh for each dataset, that would help you generate *.png or *.pfm images correspoding to different datasets.

sh predict_sf.sh
sh predict_md.sh
sh predict_kitti12.sh
sh predict_kitti15.sh

Results of our model on three benchmark datasets could also be found here

Architecture Search

Three steps for the architecture search:

1. Search

sh search.sh

2. decode

sh decode.sh

3. retrain

sh train_sf.sh
sh train_md.sh
sh train_kitti12.sh
sh train_kitti15.sh

Acknowledgements

This repository makes liberal use of code from [AutoDeeplab] [pytorch code(Non-official)].

Citing

If you find this code useful, please consider to cite our work.

@article{cheng2020hierarchical,
  title={Hierarchical Neural Architecture Search for Deep Stereo Matching},
  author={Cheng, Xuelian and Zhong, Yiran and Harandi, Mehrtash and Dai, Yuchao and Chang, Xiaojun and Li, Hongdong and Drummond, Tom and Ge, Zongyuan},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  year={2020}
}