xjg-0216/dcsnet

ICIP paper ID: 2294

Toward Snow Removal via the diversity and complexity of Snow Image

Yan Shen, Jiange Xu , Xiaotao Shao , Jinbiao Zhu and Xinmin Wang

This repo is an implementation of PyTorch version.

Image desnowing on the CSD dataset. Under different parameter capacities(x-axis), our approach performs better than other methods, as well as the stat-of-the-art(PSNR on y-axis)

Feature maps display of AFFP module:

The feature map of AFFP helps us better explain its role. Specifically, Level 1 exhibits that its ability in capturing small-scale snow elements and additional high-frequency information, and Level 3 demonstrates it is capturing large-scale snow elements and strong semantic information. By utilizing various receptive fields, our proposed AFFP can focus on snowflake elements more precisely.

The ablation experiments of the PRM module on hyperparameters are follows:

• T : the number of stages of PRM

• n : the number of CAB

T	PSNR	SSIM	GFLOPs
1	34.25	0.96	62.2
2	35.03	0.98	72.8
3	36.14	0.98	83.4
4	36.20	0.98	94.0

n	PSNR	SSIM	GFLOPs
2	34.40	0.96	75.7
4	36.14	0.98	83.4
6	36.66	0.98	91.1

Explore transferability across different datasets

The following experiments take the model trained on the CSD dataset as the baseline, and perform direct transfer or fine-tuning on SRRS/ Snow100K. Here, we benchmark against the PSNR metric.

dataset	direct transfer	fine-tuning	retrain
SRRS	30.29	30.94	31.25
Snow100K	26.80	32.89	33.64

Please note that the network structure source code will not be open source until our paper is accepted.

Installation

This implementation based on BasicSR which is a open source toolbox for image/video restoration tasks.

python 3.8.0
pytorch 1.8.0
cuda 11.1

pip install -r requirements.txt
python setup.py develop --no_cuda_ext

Quick Start (Single Image Inference)

---

• python basicsr/demo.py -opt options/demo/demo.yml

  • modified your input and output path
  • define network
  • pretrained model, it should match the define network.

Image Desnow Tasks

---

Image Desnow - CSD dataset

• prepare data

  • mkdir ./datasets/CSD

  • download dataset

  • it should be like:

    ./datasets/CSD/
    ./datasets/CSD/train/
    ./datasets/CSD/train/input/
    ./datasets/CSD/train/target/

  • python scripts/data_preparation/csd.py

    • crop the train image pairs to 256x256 patches.

• eval

  • python basicsr/test.py -opt options/test/CSD/DCSNet.yml

• train

  • python -m torch.distributed.launch --nproc_per_node=2 --master_port=4321 basicsr/train.py -opt options/train/CSD/DCSNet.yml --launcher pytorch
  • data in lmdb format will lose about 0.01 value in PSNR

dcsnet's Issues

_arch.py is not found

Hello.
I found your ICIP2022 study very interesting and tried to run this sample code.
I cloned the code and ran it as described in the README "Quick Start (Single Image Inference)", but I got an error in DCSNet/basicsr/models/archs/__init__.py.

The execution details and the error are as follows.

Disable distributed.
Traceback (most recent call last):
  File "basicsr/demo.py", line 45, in <module>
    main()
  File "basicsr/demo.py", line 39, in main
    model = create_model(opt)
  File "/home/motoya/PycharmProjects/DCSNet-master/basicsr/models/__init__.py", line 39, in create_model
    model = model_cls(opt)
  File "/home/motoya/PycharmProjects/DCSNet-master/basicsr/models/image_restoration_model.py", line 40, in __init__
    [self.net](http://self.net/)_g = define_network(deepcopy(opt['network_g']))
  File "/home/motoya/PycharmProjects/DCSNet-master/basicsr/models/archs/__init__.py", line 48, in define_network
    net = dynamic_instantiation(_arch_modules, network_type, opt)
  File "/home/motoya/PycharmProjects/DCSNet-master/basicsr/models/archs/__init__.py", line 41, in dynamic_instantiation
    if cls_ is None:
UnboundLocalError: local variable 'cls_' referenced before assignment

The error is thought to be due to the absence of _arch.py in DCSNet/basicsr/models/archs/.
Where should I get this _arch.py from?

Thank you in advance.