liuzhen03/adnet

Pytorch implementation of ADNet. (The winning method of the first edition of NTIRE2021 Multi-Frame HDR Challenge)

adnet's Issues

muPSNR too low

I use ID less than 1400 as the training set and the rest as the verification set. The PSNR on the validation set can reach 40 +, but the muPSNR is only 22. In addition, I tested that the muPSNR on 100 training sets can reach 37. The training of the model is not over, but the growth trend of muPSNR on the validation set is not obvious, which seems to be a little over fitting.

image ranges in mu loss

Hi,

Thank you for your sharing, but I am confused about your mu loss calculation. When using your mu loss, both pred and label images have pixel values larger than 1 according to your code. You do gamma correction first, which will lead to the values of hdr_linear_ref and hdr_linear_res much larger. For example, if a pixel in pred image = 10.0, it will be 174.78 after the gamma correction. Is it reasonable? Why we need to process gamma correction here? Is the gt image in NTIRE dataset in a nonlinear domain?

I quote your code of mu loss as follows.

class mu_loss(object):
    def __init__(self, gamma=2.24, percentile=99):
        self.gamma = gamma
        self.percentile = percentile

    def __call__(self, pred, label):
        hdr_linear_ref = pred ** self.gamma
        hdr_linear_res = label ** self.gamma
...

the updated results pic is broken？

Some Error

[Errno 2] No such file or directory: './data/val.npy'

I noticed in the code that you didn't provide val.npy? What is this file for? Can you provide a .py files for preprocessing?

Question regarding the use of PCD and number of DRDBs

Hi, thank you for your wonderful work in HDR imaging. In the meantime, I made an attempt to implement your algorithm. Unfortunately it was not successful. I would like to have some short questions.

• I re-use the code from EDVR Repository without any modifications (line 376-402). I concatenate LDR images to create a tensor size of b*t*c*h*w (following line 359) for input. Is this the correct use of PCD alignment?
• How many DRDBs were used? I used three blocks (same as in the AHDRNet) but the results were not good.
• Are and the results of the same convolutional filters?

Thank you for spending time reading my question.

A question on mu_loss

Hi,

Thanks for the code. I have a question on mu_loss defined here:

https://github.com/liuzhen03/ADNet/blob/main/graphs/loss/muloss.py#L21

It seems the code uses norm_perc to scale the predicted image and ground truth image for tone mapping. But norm_perc is obtained from the predicted image. Since the predicted image changes with epochs, the target that the network approximate also changes with epochs. Would it cause the training process less stable? Why not use the ground truth image to get norm_perc?

Thank you very much for your answer and help.

The PSNR-μ on Kalantari et al.'s dataset

Hi, I trained ADNET on Kalantari et al.'s dataset. However, after 200 epochs training, the PSNR-μ is about 43.8, which is lower than your results. Here are some questions:

• I see your answer in#7. Do you keep the gamma correction in mu-loss on Kalantari et al.'s dataset?

• Also, do I need special data preprocessing process?

Looking forward to your reply！

ImportError: cannot import name 'PCD_ Align 'from' dcn_ V2 '

ImportError: cannot import name 'PCD_ Align 'from' dcn_ V2 '
I encountered this issue during the debugging process. May I ask 'PCD'_ Where is the function 'Align' located?

About the visualization?

Hi, I wonder how you visualize the images in the NTIRE dataset? Thanks you!

When will the code be released?

Thanks for your contribution!
I wonder to task, when will the code be released?

can not be starred

Failed to load latest commit information.

About validation set

Hi, I am trying treproducing the artical. I randomly picked 100 scenes as the validation set, and as of now, the psnr-L is 12 db larger than psnr-mu. But in the paper, they only differ by 2 db.
I am wondering if you could kindly send me the validation set list. Looking forward to your reply.

About the validation set

Hi, thanks for your impressive work!
I am trying reproducing the artical. I randomly picked 100 scenes as the validation set, and as of now, the psnr-L is 12 db larger than psnr-mu. But in your paper, they only differ by 2 db.
I am wondering if you could kindly send me the validation set list. Looking foward to your reply.

The training setting for Kalantari et al.'s dataset?

What's the learning rate, total epochs, lr decay milestones, batch size for training Kalantari et al.'s dataset?

weight file for AHDRNet on NTIRE2021 dataset

Hi, I found that you were able to generate the results using AHDRNet. I found that AHDRNet is trained using Kalantari dataset. Would you please give me the weight file that was trained using NTIRE2021 dataset?

Thanks!

Cannot download dataset

Excuse me, I can't find the download link of the dataset you mentioned, could you please tell me how to download the dataset?

您好，在其他HDR数据集上应该怎么测试呢

我测试了AHDRNet和本文方法在 别的数据集，曝光是[1/32,1/16,1/8]这种都很诡异，是不是要调一下 LDR2HDR 的函数呢，谢谢啦

Question about inference on test images.(Memory out)

Hi.
Thanks for your works. BTW, i have question about GPU memory on test time.

I read your paper, especially about test evaluation.

The entire testing process is conducted on a single
2080Ti GPU. Constrained by the limited GPU memory, we
split each test image into size 1060x1000 and 1060x900
for testing and then concatenate them to the full size
1060x190

However, when i test on my single RTX 2080Ti GPU(12G VRAM), i faced CUDA memory out even with single PCD alignment module.
My testing code is like below.

os.environ["CUDA_VISIBLE_DEVICES"] = "3"
import torch
from dcn_v2 import PCD_Align
def requires_grad(model, flag=True):
    for p in model.parameters():
        p.requires_grad = flag
a = torch.randn([1,64,1060,900]).cuda()
b = torch.randn([1,64,530,450]).cuda()
c = torch.randn([1,64,265,225]).cuda()
model = PCD_Align().cuda()
requires_grad(model,False)

out = model([a,b,c], [a,b,c])

As you described on paper, i made tensors which have same shape of after through Pyramid module(assume original LDR image size is 1060x900). And also turned off gradient of network for saving memory.

Even i pass tensors only 1 PCD Alignment(need to run 2 times in full inference), it caused OOM on CUDA.
To make sure, i attach my GPU status with !nvidia-smi command

Thu Aug 12 22:37:59 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.80 Driver Version: 460.80 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce RTX 208... Off | 00000000:24:00.0 Off | N/A |
| 87% 80C P2 234W / 250W | 5900MiB / 11019MiB | 98% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 GeForce RTX 208... Off | 00000000:2F:00.0 Off | N/A |
| 50% 48C P8 16W / 250W | 7372MiB / 11016MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 GeForce RTX 208... Off | 00000000:30:00.0 Off | N/A |
| 99% 86C P2 196W / 250W | 10933MiB / 11019MiB | 93% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 GeForce RTX 208... Off | 00000000:31:00.0 Off | N/A |
| 39% 49C P8 12W / 250W | 8MiB / 11019MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

Can you answer this problem? It'll be great if you share some details on your test process.
BR.

Could you share your pretrained models of NTIRE and Kalantari?

Hi, @liuzhen03

Could you share your pretrained models of NTIRE and Kalantari?

Best,

data preprocess

I'm sorry to bother you. Is it convenient to provide a script for data preprocessing. To reproduce the results in Table 2 in your article. We look forward to your reply.

The performance gap between our implementation and your paper report

We just trained both ADNet and AHDRNet on NTIRE dataset, the PSNR_l is close to the result of your paper but the PSNR_mu reaches only 34 and 32, respectively. Is this caused by the different choice of validation set? Can you release your validation list?

Can you share NTIRE2021 HDR dataset to me? My email is [email protected].

Hi, When will the code be released? Thanks.

Hi, When will the code be released? Thanks.