lvfeifan/mbllen

Code for “MBLLEN: Low-light Image/Video Enhancement Using CNNs”, BMVC 2018.

MBLLEN

This is a Keras implementation of “MBLLEN: Low-light Image/Video Enhancement Using CNNs” in BMVC 2018, by Feifan Lv, Feng Lu, Jianhua Wu and Chongsoon Lim. This page provides more experiments on real low-light images compared with latest methods.

Paper and Project page

Requirements

• python 3
• Tensorflow 1.6.0
• Keras 2.2.0
• Opencv-python 3.4.2

Usage

Testing

To quickly test your own low-light images with our model, you can just run through

cd main
python test.py -i <input folder> -r <output folder> -m <model name>

By default, the code takes the data in the "../input/" folder, loads the "Syn_img_lowlight_withnoise.h5" model and saves results in the "../result/" folder. Please read the code to see other parameter settings.

Training:

First, prepare your own dataset or download our synthetic low-light dataset from our Project page. Second, change the load images path of "train.py" and "data_load.py". Then, you can just run through

cd main
python train.py

By default, the code takes the data in the "../dataset/" folder and save weights in the "./models/" folder. Please read the code to see other parameter settings.

Model

• Syn_img_lowlight_withnoise.h5 (This model is trained using synthetic lowlight images based on Poisson noise model. It is using for enhancement and denoising simultaneously.)
• Syn_img_lowlight.h5 (This model is trained using synthetic lowlight images without additional noise. It is using only for enhancement.)
• LOL_img_lowlight.h5 (This model is fine-tuned using LOL Dataset.)

Performance on Real Lowlight Images

To obtain better enhancement result, we linearly amplify the output of the network to improve contrast. Please read the code to see other parameter settings.

Performance on LOL Dataset

Our LOL fine-tuned version performs well on LOL test images.

Performance on DeepUPE Dataset

Our model is comparable with DeepUPE. Notice that, our models are not fine-tuned using DeepUPE's images (training images are not provided).

Bibtex

If you use this code for your research, please cite our paper.

@inproceedings{Lv2018MBLLEN,
 title={MBLLEN: Low-light Image/Video Enhancement Using CNNs},
 author={Feifan Lv, Feng Lu, Jianhua Wu, Chongsoon Lim},
 booktitle={British Machine Vision Conference (BMVC)},
 year={2018}
}

Related Follow-Up Work

Feifan Lv, Yu Li and Feng Lu. Attention Guided Low-light Image Enhancement with a Large Scale Low-light Simulation Dataset. IJCV, 2021. Paper and Project page

mbllen's Issues

GPU or CPU

这个模型是用CPU训练的吗？ 想用GPU的话从哪里该改呢？？

似乎region loss部分存在错误

您好，我想请，问在这里写了负号，求得的阈值应该也是负的，为什么后边就直接拿这个负阈值作为了判断条件？

关于新工作Attention-guided Low-light Image Enhancement的一个问题

作者你好！我关注了你光照增强的一个新工作Attention-guided Low-light Image Enhancement，冒昧在这里请教一下。
在合成低光照数据时，你给了两条公式，一条在做伽马变换，一条添加噪声，请问这是两个具有先后顺序的步骤吗？还是完全无关的？因为看到公式中的输入都I_{in}产生了疑惑。

.h5文件怎么打开

.h5文件怎么打开

Any plan on releasing the video version ?

MBLLVEN

Where can I find the video version (MBLLVEN) model?

Code for Low-light image synthesis

The test results are very good. But I can't find the code for generating Low-light image synthesis from the original image in the repo.

Could you please share the code for generating Low-light image synthesis?

thx

EM implementation

Hi, I have a question about the EM module implementation in the code. In the paper, the EM module is described as having the kernel numbers of 8, 16,16, 16, 16, 8, 3 (in total of 7 layers). But in the code implementation, it only has 6 layers has the kernel numbers of 8,8,16,32,16,8,3, which is different than the description in the original paper. I am wondering is there any specific reason about why choosing the 6 layers implementation in the code instead of the 7 layers described in the paper? Thanks

AttributeError: 'str' object has no attribute 'decode'

solution：
pip install h5py==2.10.0

How long for training

Hello, authors! I have a doubt about how long it takes to converge your model. Your manuscript mentions that "The proposed MBLLEN can be quickly converged after being trained for 5000 mini-batches". However, your train.py sets 200 epochs with 200 batches for each epoch. Therefore, the total mini-batches should be 40,000 instead of 5,000. Hope you can clarify it. Thanks!

关于网络输出结果

作者你好，我观察到网络的输出结果有层雾蒙蒙的褪色感觉，需要做后处理才能得到正常的结果。我想请假下，训练模型时都是采用正常的数据，为什么网络直接出来的结果会有雾蒙蒙的感觉呢？

No error no output :(

(mbllen-py) D:\MBLLEN-LowLightImageVideoEnhancement\MBLLEN-winpjt\main>python test.py -i /input/inputtest -m Syn_img_lowlight_withnoise
Using TensorFlow backend.
2020-04-21 09:29:27.820021: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-04-21 09:29:27.959053: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1212] Found device 0 with properties:
name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
totalMemory: 6.00GiB freeMemory: 4.97GiB
2020-04-21 09:29:27.968401: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1312] Adding visible gpu devices: 0
2020-04-21 09:29:28.464909: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4744 MB memory) -> p
hysical GPU (device: 0, name: GeForce GTX 1060, pci bus id: 0000:01:00.0, compute capability: 6.1)

I put test image in folder D:\MBLLEN-LowLightImageVideoEnhancement\MBLLEN-winpjt\input\inputtest

python has created the new folder result, but no output image created, no error!

Dataset

Hi,
It is hard to download dataset because it requires a Baidu Account and understanding of Mandarin. Would it be possible to email the datasets/post in english facing site?

is the code gray_fake_B define false？

the normal gray = R*0.3 + G*0.59 + B*0.11，
but in test.py，gray_fake_B = fake_B[:, :, 0] * 0.299 + fake_B[:, :, 1] * 0.587 + fake_B[:, :, 1] * 0.114
is the fake_B[:, :, 1] * 0.114 should be fake_B[:, :, 2] * 0.114?

Unable to train

I'm unable to train the model. Its stuck for ever as shown in this pic. I tried reducing the learning rate, reduced the training samples, epoch =2. its not even completing 1 epoch

数据集问题

项目地址失效了http://phi-ai.buaa.edu.cn/project/MBLLEN/default.htm

跑通代码？

是否有人跑通了这个代码？希望各位大佬能帮帮忙，可能是由于我的版本问题一直在报错，无法跑通模型，报错情况如下：
Using TensorFlow backend.
Traceback (most recent call last):
File "D:\Literature\Code\MBLLEN\MBLLEN\main\test.py", line 31, in
mbllen = Network.build_mbllen((None, None, 3))
File "D:\Literature\Code\MBLLEN\MBLLEN\main\Network.py", line 24, in build_mbllen
if self.bias:
File "E:\Python\anaconde1\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 498, in bool
return bool(self.read_value())
File "E:\Python\anaconde1\lib\site-packages\tensorflow\python\framework\ops.py", line 900, in bool
self._disallow_bool_casting()
File "E:\Python\anaconde1\lib\site-packages\tensorflow\python\framework\ops.py", line 507, in _disallow_bool_casting
self._disallow_in_graph_mode("using a tf.Tensor as a Python bool")
File "E:\Python\anaconde1\lib\site-packages\tensorflow\python\framework\ops.py", line 494, in _disallow_in_graph_mode
raise errors.OperatorNotAllowedInGraphError(
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

Allocation of 509607936 exceeds 10% of free system memory.

Hi!
I am getting this error Allocation of 509607936 exceeds 10% of free system memory. when I try to test this using test.py. My input folder also only contains 4 images. Can anyone help me out?

训练数据集问题

the code for calculating variety of different metrics, including AB，VIF，LOE，TMQI

Can you provide the code for calculating various metrics？Did you retrain all models and the calculate metrics?

pytorch

Can you share the code of pytorch?

When deos video vision come ?

I am interested in your project ? I wanna to know when video vision come?

请问可以提供训练数据嘛

Code for AgLLNet

Hello,
I find you guys have already improved over the MBLLEN model with AgLLNet.
Although the paper on AgLLNet was published back in 2019, I believe the code has not been released yet.

Do you intend to release it soon?

ValueError: unsupported format character ',' (0x2c) at index 42

how to handle?

When Training, loss and bright_mse is NAN and bright_psnr is INF

When Training, it shows loss: 0.0000e+00 - bright_mae: 0.0000e+00 - bright_mse: 0.0000e+00 - bright_psnr: inf - bright_SSIM: 1.0000 - bright_AB: 0.0000e+00 - LR = 0.0009900001
The loss is very strange, and I don't know why?
Then Batch 0: Invalid loss, terminating training

Training won't initiate

Hello,

the inference works perfectly but I'm having some problem while training. I've succesfully set-up the training environment (training data/params.). But once I try to train the model, the script prints out the following and seems like it gets stuck:

Model: "model_2"

---

Layer (type) Output Shape Param # Connected to

input_2 (InputLayer) [(None, 256, 256, 6)] 0 []

lambda (Lambda) (None, 256, 256, 3) 0 ['input_2[0][0]']

model (Functional) (None, 256, 256, 3) 450171 ['lambda[0][0]']

lambda_1 (Lambda) (None, 256, 256, 3) 0 ['input_2[0][0]']

lambda_2 (Lambda) (None, 256, 256, 3) 0 ['model[0][0]']

lambda_4 (Lambda) (None, 256, 256, 3) 0 ['lambda_1[0][0]']

model_1 (Functional) (None, None, None, 256) 2325568 ['lambda_2[0][0]',
'lambda_4[0][0]']

lambda_3 (Lambda) (None, 256, 256, 16) 0 ['model_1[0][0]']

lambda_5 (Lambda) (None, 256, 256, 16) 0 ['model_1[1][0]']

concatenate_9 (Concatenate (None, 256, 256, 35) 0 ['model[0][0]',
) 'lambda_3[0][0]',
'lambda_5[0][0]']

==================================================================================================
Total params: 2775739 (10.59 MB)
Trainable params: 450171 (1.72 MB)
Non-trainable params: 2325568 (8.87 MB)

---

Is it because the Network is very big that it takes too much time and I have to wait? I've also tried training on Colab.

ERROR issues

TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'