yunguo224/lhp-rain

[ICCV 2023] Official repository for the paper "From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal".

From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal (ICCV 2023)

Yun Guo^, Xueyao Xiao^, Yi Chang*, Shumin Deng, Luxin Yan

Paper link: [arxiv] [ICCV]

Project website: [link] (Benchmark available now!)

---

Learning-based image deraining methods have made great progress. However, the lack of large-scale high-quality paired training samples is the main bottleneck to hamper the real image deraining (RID). To address this dilemma and advance RID, we construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset over the existing ones are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher quality ground-truth. Specifically, the real rains in LHP-Rain not only contain the classical rain streak/veiling/occlusion in the sky, but also the splashing on the ground overlooked by deraining community. Moreover, we propose a novel robust low-rank tensor recovery model to generate the GT with better separating the static background from the dynamic rain. In addition, we design a simple transformer-based single image deraining baseline, which simultaneously utilize the self-attention and cross-layer attention within the image and rain layer with discriminative feature representation. Extensive experiments verify the superiority of the proposed dataset and deraining method over state-of-the-art.

---

Benchmark Download

We provide full version, simple version and high-level annotations of LHP-Rain. The benchmark has been updated in Project website.

Package dependencies

The project is built with PyTorch 1.9.0, Python3.7, CUDA11.1. For package dependencies, you can install them by:

pip install -r requirements.txt

Training

To train SCD-Former, you can begin the training by:

python train/train_derain.py --arch Uformer_B --batch_size 8 --gpu '0,1' --train_ps 256 --train_dir ./train --val_ps 256 --val_dir ./test --env _derain --nepoch 3000 --checkpoint 500 --warmup

Evaluation

To evaluate SCD-Former, you can run:

python test_derain.py

Citation

If you find this project useful in your research, please consider citing:

@InProceedings{Guo_2023_ICCV,
    author    = {Guo, Yun and Xiao, Xueyao and Chang, Yi and Deng, Shumin and Yan, Luxin},
    title     = {From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {12097-12107}
}

Acknowledgement

The code of SCD-Former is based on Uformer.

Contact

Please contact us if there is any question or suggestion(Yun Guo [email protected], Yi Chang [email protected]).

lhp-rain's Issues

Confusions about the datasets.

1. I have noticed that the proposed method (presented in Figure 7 of the manuscript), which was trained on the LHP-Rain datasets, still struggles to effectively handle hazy-like effects. Additionally, the proposed pipeline for obtaining clean images (GT) appears to be insufficient in suppressing veiling effects.

2. The proposed dataset is extensive, diverse, and provides significant benefits to the research community. However, should we consider conducting additional experiments to determine whether the dataset is overly redundant? It is worth noting that for each scene, a single clean image corresponds to approximately 300+ rainy images.

复现论文

请问在LHP数据集上复现论文时，训练超参数batch size、epoch数等应该怎么选取？训练集是使用train-simple还是Full？测试集是使用train-simple还是Full？

pretrained weights

Thanks for your nice work.

I want to know if the trained weights are available?