hli1221/imagefusion_deepfuse

Image fusion based on deepfuse network - Tensorflow (based on ICCV2017: deepfuse), Unofficial

Deepfuse - Tensorflow

Python>=3.0, TensorFlow=1.8.0

The loss function and training strategy are differences whit DeepFuse. I will fix these in the near feature.

This code is based on K. Ram Prabhakar, V Sai Srikar, R. Venkatesh Babu. DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs. ICCV2017, pp. 4714-4722

In this code, for all conv layers, the filter size is 3*3. And this code is not a complete version for DeepFuse, we just implement one channel fusion method which use CNN network.

This code is not exactlly same with paper in ICCV2017. The aim of the training process is to reconstruct the input image by this network. The encoder(C1, C2) is used to extract image features and decoder(C3, C4, C5) is a reconstruct tool. The fusion strategy( Tensor addition) is only used in testing process.

We train this network using Microsoft COCO dataset(T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 3-5.) as input images which contains 80000 images and all resize to 256×256 and RGB images are transformed to gray ones.

Citation

@misc{li2017deepfuse,
    author = {Hui Li},
    title = {CODE: Image Fusion based on Deepfuse},
    year = {2018},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/hli1221/Imagefusion_deepfuse}}
  }

imagefusion_deepfuse's Issues

How to transform to result of RGB color images for multi-exposure experimentation.

Hollow, Li Hui.

Currently I'm studying about the multi-exposure image fusion.
So, I want to experiment with your code...
But, your result of experiment is only gray image ones.
Please help me, where i change your code for fused RGB color images that are multi-exposured imaged.

There are differences between SSIM LOSS and MEF SSIM LOSS

hey there, I notice that your code for loss function in ssim_loss_function.py is quite similar to the code I found in https://stackoverflow.com/questions/39051451/ssim-ms-ssim-for-tensorflow

However, this seems to be SSIM LOSS function instead of MEF SSIM LOSS function. In
K. Ma, K. Zeng, and Z. Wang. Perceptual quality assessment for multi-exposure image fusion. IEEE Transactions on Image Processing, 24(11):3345–3356, 2015. , the author mentioned that "Direct use of the SSIM algorithm [27], however, is impossible, which requires a single perfect quality reference image. "

The paper Deepfuse's work is exactly on MEF(multi-exposure image fusion), so I would think there should be differences between SSIM LOSS and MEF SSIM LOSS, in other words, the loss function in your code may be incorrect?

Look forward to your reply, thx