yunguo224/nlcl

[CVPR 2022] Official repository for the paper "Unsupervised Deraining Where Contrastive Learning Meets Self-similarity".

Unsupervised Deraining Where Contrastive Learning Meets Self-similarity (CVPR 2022)

Yuntong Ye, Changfeng Yu, Yi Chang, Lin Zhu, Xi-le Zhao, Luxin Yan, Yonghong Tian

Paper link: [Arxiv] [CVPR]

---

In this work, we propose a novel non-local contrastive learning (NLCL) method for unsupervised image deraining. Consequently, we not only utilize the intrinsic self-similarity property within samples, but also the mutually exclusive property between the two layers, so as to better differ the rain layer from the clean image. Specifically, the non-local self-similarity image layer patches as the positives are pulled together and similar rain layer patches as the negatives are pushed away. Thus the similar positive/negative samples that are close in the original space benefit us to enrich more discriminative representation. Apart from the self-similarity sampling strategy, we analyze how to choose an appropriate feature encoder in NLCL. Extensive experiments on different real rainy datasets demonstrate that the proposed method obtains state-of-the-art performance in deraining.

Package dependencies

The project is built with PyTorch 1.6.0, Python3.6. For package dependencies, you can install them by:

pip install -r requirements.txt

Pretrained model

The pre-trained models of both Rain and Background Generator Networks are provided in checkpoints/RealRain.

Training

To train NLCL on real rain dataset, you can begin the training by:

python train.py --dataroot DATASET_ROOT --model NLCL --name NAME --dataset_mode unaligned

The DATASET_ROOT example are provided in datasets/RealRain.

Evaluation

To evaluate NLCL, you can run:

python test.py --dataroot DATASET_ROOT --model NLCL --name NAME --dataset_mode single --preprocess None

Citation

If you find this project useful in your research, please consider citing:

@InProceedings{Ye_2022_CVPR,
    author    = {Ye, Yuntong and Yu, Changfeng and Chang, Yi and Zhu, Lin and Zhao, Xi-Le and Yan, Luxin and Tian, Yonghong},
    title     = {Unsupervised Deraining: Where Contrastive Learning Meets Self-Similarity},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {5821-5830}
}

Acknowledgement

This code is inspired by CycleGAN.

Contact

Please contact us if there is any question or suggestion(Yun Guo [email protected], Yuntong Ye [email protected], Yi Chang [email protected]).

nlcl's Issues

论文复现

请问是否能够给出论文中结果所对应的训练配置以及所使用的数据集？

Some problems about 实验中batchsize的设置

1. 您好, 您给的代码中batchsize是1, 我可以跑通你的代码, 但是我发现如果将batchsize设为其他数值,该实验将会报错, 具体错误是在networks.py中的PatchSampleNonlocalOneGroup函数中 diff_pow = torch.pow((patches - sample_patch), 2) 这一行, 我看了错误, 具体原因是 batchsize设为别的数值, sample_patch在第一个维度上无法广播到和patches一样的维度。想问下您知道这个的解决方法是什么呢？

关于实验中使用的数据集的疑问

您好, 我想问一下实验中所使用的trainA和trainB有什么关系, cyclegan里面的trainA和trainB分别是有雨图和unpair的无雨图, 但是我不太理解你实验中的trainA和trainB有什么关系。

DisNCE loss

Why is the value of my DisNCE loss consistently unchanged at 4.674? Have you encountered a similar issue before?

About checkpoint files

Hello, I have read your paper and I found that it is really interesting.

I am curious that which dataset did you use in training the checkpoint files in checkpoint directory? (latest_net_Back.pth, latest_net_Rain.pth)

Did you use the dataset in datasets/RealRain directory?

If so, then can you provide another checkpoints from another datasets (e.g. SPA-DATA, RainCityscape)

Thanks in advance!

Error

Hello, thanks for sharing the excellent work. I meet some error.

1. File "NLCL-main/models/patchnce.py", line 34, in forward
   if self.opt.nce_includes_all_negatives_from_minibatch:
   AttributeError: 'Namespace' object has no attribute 'nce_includes_all_negatives_from_minibatch'

2. ModuleNotFoundError: No module named 'data.single_dataset' when I try to test it.