jxzhangjhu/weatherdiffusion

Code for "Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models" [arXiv preprint 2207.14626, 2022]

Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models

This is the code repository of the following paper to train and perform inference with patch-based diffusion models for image restoration under adverse weather conditions.

"Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models"
Ozan Özdenizci, Robert Legenstein
arXiv preprint arXiv:2207.14626 (2022).

Currently the repository is still being prepared, more details to be included soon...

In the meantime, check out below for some visualizations of our patch-based diffusive image restoration approach.

Image Desnowing

Input Condition	Restoration Process	Output

Image Deraining & Dehazing

Input Condition	Restoration Process	Output

Raindrop Removal

Input Condition	Restoration Process	Output

Datasets

We perform experiments for image desnowing on Snow100K, combined image deraining and dehazing on Outdoor-Rain, and raindrop removal on the RainDrop datasets. To train multi-weather restoration, we used the AllWeather training set from TransWeather, which is composed of subsets of training images from these three benchmarks.

Reference

If you use this code or models in your research and find it helpful, please cite the following paper:

@article{ozdenizci2022,
  title={Restoring vision in adverse weather conditions with patch-based denoising diffusion models},
  author={Ozan \"{O}zdenizci and Robert Legenstein},
  journal={arXiv preprint arXiv:2207.14626},
  year={2022}
}

Acknowledgments

Authors of this work are affiliated with Graz University of Technology, Institute of Theoretical Computer Science, and Silicon Austria Labs, TU Graz - SAL Dependable Embedded Systems Lab, Graz, Austria. This work has been supported by the "University SAL Labs" initiative of Silicon Austria Labs (SAL) and its Austrian partner universities for applied fundamental research for electronic based systems.

Parts of this code repository is based on the following works:

• https://github.com/ermongroup/ddim
• https://github.com/bahjat-kawar/ddrm
• https://github.com/JingyunLiang/SwinIR