A Lightweight Self-supervised Depth Estimation Model featuring Dynamic Upsampling and Triplet Loss Optimization
Hao Jiang
Xiaoyan Jiang
This work was supported by National Natural Science Foundation of China (NSFC) Essential project (No. U2033218)
The paper was accepted by ICPR 2024 (27th International Conference on Pattern Recognition)
(LiDUT-Depth 1024x320)
You can download the trained models using the links below.
Model | Params | ImageNet Pretrained | Input size | Abs Rel | Sq Rel | RMSE | RMSE log | delta < 1.25 | delta < 1.25^2 | delta < 1.25^3 |
---|---|---|---|---|---|---|---|---|---|---|
LiDUT-Depth_640x192 | 3.1M | yes | 640x192 | 0.103 | 0.723 | 4.469 | 0.178 | 0.889 | 0.964 | 0.984 |
LiDUT-Depth_1024x320 | 3.1M | yes | 1024x320 | 0.101 | 0.703 | 4.354 | 0.175 | 0.895 | 0.966 | 0.985 |
Assuming a fresh Anaconda distribution, you can install the dependencies with:
conda create -n lidut-depth python=3.10
conda activate lidut-depth
pip install -r requirements.txt
pip install 'git+https://github.com/saadnaeem-dev/pytorch-linear-warmup-cosine-annealing-warm-restarts-weight-decay'
We ran our experiments with PyTorch 2.0.1, CUDA 11.8, Python 3.10.12 and Ubuntu 20.04.
We recommend using a conda environment to avoid dependency conflicts.
You can download the entire raw KITTI dataset by running:
wget -i splits/kitti_archives_to_download.txt -P kitti_data/
Then unzip with
cd kitti_data
unzip "*.zip"
cd ..
Warning: it weighs about 175GB, so make sure you have enough space to unzip too!
Our default settings expect that you have converted the png images to jpeg with this command, which also deletes the raw KITTI .png
files:
find kitti_data/ -name '*.png' | parallel 'convert -quality 92 -sampling-factor 2x2,1x1,1x1 {.}.png {.}.jpg && rm {}'
or you can skip this conversion step and train from raw png files by adding the flag --png
when training, at the expense of slower load times.
Pre-computed segmentation images provided by FSRE-Depth are also needed for training.
Note: The pre-computed segmentation is NOT needed for evaluating.
KITTI-DATA/
├── 2011_09_26/
├── 2011_09_28/
├── 2011_09_29/
├── 2011_09_30/
├── 2011_10_03/
└── segmentation/ # download and unzip "segmentation.zip"
python test_simple.py --load_weights_folder path/to/your/weights/folder --image_path path/to/your/test/image
To prepare the ground truth depth maps run:
python export_gt_depth.py --data_path kitti_data --split eigen
python export_gt_depth.py --data_path kitti_data --split eigen_benchmark
...assuming that you have placed the KITTI dataset in the default location of ./kitti_data/
.
Evaluate the weights of a single epoch:
python evaluate_depth.py --load_weights_folder path/to/your/weights/folder --data_path path/to/kitti_data/
Or evaluate the weights of every epochs together (will print all the evaluate results to a txt file in your model folder):
python evaluate_depth_all.py --load_weights_folder path/to/your/model/folder --data_path path/to/kitti_data/ --num_epochs number/of/your/epochs/to/evaluate
To evaluate the model's generalization ability, you'll need to prepare the Make3D dataset first. Download Test134 images and depths, simply uncompress them to ./splits/make3d
, then run ./splits/make3d/make3d_extraction.py
to get the image filenames, finally run ./evaluate_depth_make3d.py
to get the generalization ability result (you may need to modify the data path in the code):
python evaluate_depth_make3d.py
python train.py --data_path path/to/your/data --model_name test --num_epochs 30 --batch_size 12 --num_workers 16 --mypretrain path/to/your/pretrained/weights
tensorboard --log_dir ./tmp/mytrain
# coming online soon