This is the official code of SDAP: Random Sub-Samples Generation for Self-Supervised Real Image Denoising.
With sufficient paired training samples, the supervised deep learning methods have attracted much attention in image denoising because of their superior performance. However, it is still very challenging to widely utilize the supervised methods in real cases due to the lack of paired noisy-clean images. Meanwhile, most self-supervised denoising methods are ineffective as well when applied to the real-world denoising tasks because of their strict assumptions in applications. For example, as a typical method for self-supervised denoising, the original blind spot network (BSN) assumes that the noise is pixel-wise independent, which is much different from the real cases. To solve this problem, we propose a novel self-supervised real image denoising framework named Sampling Difference As Perturbation (SDAP) based on Random Sub-samples Generation (RSG) with a cyclic sample difference loss. Specifically, we dig deeper into the properties of BSN to make it more suitable for real noise. Surprisingly, we find that adding an appropriate perturbation to the training images can effectively improve the performance of BSN. Further, we propose that the sampling difference can be considered as perturbation to achieve better results. Finally we propose a new BSN framework in combination with our RSG strategy. The results show that it significantly outperforms other state-of-the-art self-supervised denoising methods on real-world datasets.
Our experiments are done with:
- Python 3.7.9
- PyTorch 1.7.1
- numpy 1.19.5
- opencv 4.5.1
- scikit-image 0.17.2
| Training Dataset | Model |
|---|---|
| SIDD Medium | SDAP.pth |
| SIDD Medium + SIDD Benchmark | SDAP_S_for_SIDD.pth |
| SIDD Medium + DND Benchmark | SDAP_S_for_DND.pth |
You can get the complete SIDD validation dataset from https://www.eecs.yorku.ca/~kamel/sidd/benchmark.php.
'.mat' files need to be converted to images ('.png').
run test.pyto output the denoising results of our proposed method.
'.mat' files in training datasets need to be converted to images ('.png').
You have to add the noisy images ('.png') for training in './dataset/train_data'.
Tip: Reducing the size of the training image speeds up the reading of the dataset thus speeding up training. So you can crop the image to a fixed size (e.g. 512*512) in advance.
run train.py.
@inproceedings{Pan_2023_ICCV,
author = {Pan, Yizhong and Liu, Xiao and Liao, Xiangyu and Cao, Yuanzhouhan and Ren, Chao},
title = {Random Sub-Samples Generation for Self-Supervised Real Image Denoising},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {12150-12159}
}
If you have any questions, please contact [email protected].
The codes are based on AP-BSN and Neighbor2Neighbor. Thanks for their awesome works.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent