fengzhang427/hep

Official implementation for “Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior”

HEP

Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior

Implementation

• Python3
• PyTorch>=1.0
• NVIDIA GPU+CUDA

Training process

The original LOL dataset can be downloaded from here. The EnlightenGAN dataset can be downloaded from here Before starting training process, you should modify the data_root in ./config, and then run the following command

python LUM_train.py
python NDM_train.py

Testing process

Please put test images into 'test_images' folder and download the pre-trained checkpoints from google drive(put it into ./checkpoints), then just run

python NDM_test.py

You can also just evaluate the stage one (LUM), just run

python LUM_test.py

Paper Summary

HEP consists of two stages, Light Up Module (LUM) and Noise Disentanglement Module (LUM)

Representative Visual Results

More visual results can be found in asssets.

Citation

if you find this repo is helpful, please cite

@article{zhang2021unsupervised,
  title={Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior},
  author={Zhang, Feng and Shao, Yuanjie and Sun, Yishi and Zhu, Kai and Gao, Changxin and Sang, Nong},
  journal={arXiv preprint arXiv:2112.01766},
  year={2021}
}

hep's Issues

can you provide the source code

Normalization in VGG preprocess

Hi, as stated in torchvision page, the input to torchvision's pretrained VGG should be RGB format and normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225].

However in your code, RGB image is converted to BGR format and normalized by mean=[103.939, 116.779, 123.680] and std=[1.0, 1.0, 1.0]:

HEP/utils.py

Lines 219 to 229 in c0188bb

	def vgg_preprocess(batch):
	tensor_type = type(batch.data)
	(r, g, b) = torch.chunk(batch, 3, dim=1)
	batch = torch.cat((b, g, r), dim=1) # convert RGB to BGR
	batch = batch * 255 # * 0.5 [-1, 1] -> [0, 255]
	mean = tensor_type(batch.data.size()).cuda()
	mean[:, 0, :, :] = 103.939
	mean[:, 1, :, :] = 116.779
	mean[:, 2, :, :] = 123.680
	batch = batch.sub(Variable(mean)) # subtract mean
	return batch

Should this be fixed?

Dimension mismatch for x2 and x5 in LUM_model.py

Traceback (most recent call last):
File "LUM_train.py", line 156, in
main()
File "LUM_train.py", line 134, in main
r_x, i_x = light(val_x)
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/deepakpanda4/code/Users/deepakpanda/HEP/models/LUM_model.py", line 48, in forward
cat5 = torch.cat((x5, x2), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 766 but got size 765 for tensor number 1 in the list.

Question about using reflectance map as the output

Hi, the HE prior is awesome, but I am kind of confused about directly using (denoised) reflectance map as the final output, without combined with the (enhanced) illumination map. In most low-light enhancement methods, to my knowledge, they separately enhance the reflectance map and the illumination map, and multiply them to construct the final enhanced image. I am wondering if it is a intended design? Will the output be over-brighten?

Applying to a general image restoration

Hello
How are you?
Thanks for contributing to this project.
I am going to use this method for a general image restoration (low quality -> high quality).
There are several kinds of images such as blur and noise samples with noise in the low quality images.
Is it possible to apply this method without any architecture change to such a general image restoration task?
Thanks

Question about train NDM model

When running NDM_train.py, the graphics card will stop occupying 60-200+ iterations each time, and there will be no more output, but the program will not actively report an error or stop. At present, I have not made other modifications to the code. LOL is selected as the training set, with a total of 485 images. I would like to ask if other settings are required for the code, if there are any requirements for training sets, graphics cards, etc., and if some other parameters need to be modified. Thank you!