limbee/ntire2017

Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

ntire2017's Issues

about PSNR and SSIM on unknown track

Hi~Thanks for your superior works. I want to know the PSNR and SSIM that you achvevd on the unknown track of DIV2K dataset. Thank you.

Which set used in Table 2 of paper?

I am having a slight confusion regarding the dataset used for Table 2 written in paper.

The figure caption reads: "Performance comparison between architectures on the DIV2K validation set". The DIV2K validation consists of 100 images (800-900)
However, in section 4.4, it is stated that, "Evaluation is conducted on the 10 images of
DIV2K validation set"

I have a confusion:
Q. 'DIV2K validation set' refer to which set? The actual validation set consisting of 100 images or your custom validation set you created (10 images) during training?

Also, kindly mention the file-names of images used in Table2 computation, thanks.

Single channel picture in set14

There is a picture (bridge) in set 14 that is single-channel, but the trained model requires that the input picture be three-channel. How did you solve this problem in the experiment?

How can I test these models in the CPU-only environment?

Hi LimBee,
For some reasons,I can train the models with GPUs,but I have to test the pre-trained model in the CPU-only environment.Since I is not really good at Torch,I will be very appreciate for your help.

Are there any other framework implementation for this paper?

Such as tensorflow or keras, I think your EDSR model isn't too complicated to implement. I'm interested in re-implementing it in keras.

Process is eventually killed

Iterations start taking more time and eventually the process is killed automatically. Any idea why this might be happening? Usually the process terminates somewhere around 11th epoch. Output of the program is attached below:

Creating data loader...	
loading data...	
	Initializing data loader for train set...	
	Initializing data loader for val set...	
Train start	
[Iter: 0.1k / lr: 1.00e-4] 	Time: 59.98 (Data: 49.06) 	Err: 33.426155	
[Iter: 0.2k / lr: 1.00e-4] 	Time: 58.74 (Data: 50.61) 	Err: 17.742222	
[Iter: 0.3k / lr: 1.00e-4] 	Time: 57.86 (Data: 49.84) 	Err: 15.524932	
[Iter: 0.4k / lr: 1.00e-4] 	Time: 58.06 (Data: 49.76) 	Err: 13.206391	
[Iter: 0.5k / lr: 1.00e-4] 	Time: 59.22 (Data: 50.93) 	Err: 11.778802	
[Iter: 0.6k / lr: 1.00e-4] 	Time: 58.67 (Data: 49.99) 	Err: 11.223848	
[Iter: 0.7k / lr: 1.00e-4] 	Time: 57.71 (Data: 49.69) 	Err: 10.476344	
[Iter: 0.8k / lr: 1.00e-4] 	Time: 58.65 (Data: 50.37) 	Err: 10.352445	
[Iter: 0.9k / lr: 1.00e-4] 	Time: 58.19 (Data: 49.89) 	Err: 9.883691	
[Iter: 1.0k / lr: 1.00e-4] 	Time: 58.66 (Data: 50.56) 	Err: 9.862550	
[Epoch 1 (iter/epoch: 1000)] Test time: 23.26	
(scale 4) Average PSNR: 27.0086 (Highest ever: 27.0086 at epoch = 1)	
	
[Iter: 1.1k / lr: 1.00e-4] 	Time: 59.21 (Data: 50.59) 	Err: 9.435181	
[Iter: 1.2k / lr: 1.00e-4] 	Time: 57.73 (Data: 49.44) 	Err: 9.251034	
[Iter: 1.3k / lr: 1.00e-4] 	Time: 58.37 (Data: 50.18) 	Err: 9.082382	
[Iter: 1.4k / lr: 1.00e-4] 	Time: 58.64 (Data: 50.27) 	Err: 8.877250	
[Iter: 1.5k / lr: 1.00e-4] 	Time: 57.33 (Data: 49.34) 	Err: 8.786548	
[Iter: 1.6k / lr: 1.00e-4] 	Time: 58.76 (Data: 50.61) 	Err: 8.352576	
[Iter: 1.7k / lr: 1.00e-4] 	Time: 57.96 (Data: 49.72) 	Err: 8.513297	
[Iter: 1.8k / lr: 1.00e-4] 	Time: 57.93 (Data: 49.91) 	Err: 8.242201	
[Iter: 1.9k / lr: 1.00e-4] 	Time: 57.72 (Data: 49.67) 	Err: 8.255659	
[Iter: 2.0k / lr: 1.00e-4] 	Time: 58.26 (Data: 50.18) 	Err: 8.005295	
[Epoch 2 (iter/epoch: 1000)] Test time: 23.43	
(scale 4) Average PSNR: 27.5600 (Highest ever: 27.5600 at epoch = 2)	
	
[Iter: 2.1k / lr: 1.00e-4] 	Time: 59.21 (Data: 50.80) 	Err: 8.167938	
[Iter: 2.2k / lr: 1.00e-4] 	Time: 58.26 (Data: 49.71) 	Err: 8.067993	
[Iter: 2.3k / lr: 1.00e-4] 	Time: 58.07 (Data: 49.75) 	Err: 8.067032	
[Iter: 2.4k / lr: 1.00e-4] 	Time: 57.78 (Data: 49.53) 	Err: 8.029886	
[Iter: 2.5k / lr: 1.00e-4] 	Time: 58.21 (Data: 50.22) 	Err: 8.389706	
[Iter: 2.6k / lr: 1.00e-4] 	Time: 58.52 (Data: 50.37) 	Err: 8.040120	
[Iter: 2.7k / lr: 1.00e-4] 	Time: 57.95 (Data: 49.76) 	Err: 7.712390	
[Iter: 2.8k / lr: 1.00e-4] 	Time: 58.30 (Data: 50.30) 	Err: 8.082315	
[Iter: 2.9k / lr: 1.00e-4] 	Time: 58.21 (Data: 50.17) 	Err: 7.580434	
[Iter: 3.0k / lr: 1.00e-4] 	Time: 58.14 (Data: 49.73) 	Err: 7.440087	
[Epoch 3 (iter/epoch: 1000)] Test time: 23.53	
(scale 4) Average PSNR: 27.8171 (Highest ever: 27.8171 at epoch = 3)	
	
[Iter: 3.1k / lr: 1.00e-4] 	Time: 59.66 (Data: 50.99) 	Err: 7.984920	
[Iter: 3.2k / lr: 1.00e-4] 	Time: 57.77 (Data: 49.08) 	Err: 7.743653	
[Iter: 3.3k / lr: 1.00e-4] 	Time: 58.84 (Data: 50.73) 	Err: 7.766374	
[Iter: 3.4k / lr: 1.00e-4] 	Time: 58.35 (Data: 49.99) 	Err: 7.688996	
[Iter: 3.5k / lr: 1.00e-4] 	Time: 59.49 (Data: 50.64) 	Err: 7.697878	
[Iter: 3.6k / lr: 1.00e-4] 	Time: 58.05 (Data: 49.94) 	Err: 7.589858	
[Iter: 3.7k / lr: 1.00e-4] 	Time: 57.57 (Data: 49.55) 	Err: 7.795816	
[Iter: 3.8k / lr: 1.00e-4] 	Time: 57.88 (Data: 49.92) 	Err: 7.386197	
[Iter: 3.9k / lr: 1.00e-4] 	Time: 58.64 (Data: 50.19) 	Err: 7.491277	
[Iter: 4.0k / lr: 1.00e-4] 	Time: 57.93 (Data: 49.67) 	Err: 7.493202	
[Epoch 4 (iter/epoch: 1000)] Test time: 23.77	
(scale 4) Average PSNR: 27.9856 (Highest ever: 27.9856 at epoch = 4)	
	
[Iter: 4.1k / lr: 1.00e-4] 	Time: 58.35 (Data: 50.14) 	Err: 7.274099	
[Iter: 4.2k / lr: 1.00e-4] 	Time: 59.01 (Data: 50.77) 	Err: 7.444119	
[Iter: 4.3k / lr: 1.00e-4] 	Time: 57.99 (Data: 49.68) 	Err: 7.388358	
[Iter: 4.4k / lr: 1.00e-4] 	Time: 58.33 (Data: 50.28) 	Err: 7.441720	
[Iter: 4.5k / lr: 1.00e-4] 	Time: 58.04 (Data: 49.80) 	Err: 7.664649	
[Iter: 4.6k / lr: 1.00e-4] 	Time: 57.90 (Data: 49.77) 	Err: 7.053351	
[Iter: 4.7k / lr: 1.00e-4] 	Time: 58.90 (Data: 50.54) 	Err: 7.477736	
[Iter: 4.8k / lr: 1.00e-4] 	Time: 57.74 (Data: 49.68) 	Err: 7.428355	
[Iter: 4.9k / lr: 1.00e-4] 	Time: 58.28 (Data: 50.14) 	Err: 7.505870	
[Iter: 5.0k / lr: 1.00e-4] 	Time: 58.14 (Data: 49.78) 	Err: 7.511303	
[Epoch 5 (iter/epoch: 1000)] Test time: 23.88	
(scale 4) Average PSNR: 28.1898 (Highest ever: 28.1898 at epoch = 5)	
	
[Iter: 5.1k / lr: 1.00e-4] 	Time: 59.04 (Data: 50.48) 	Err: 7.640118	
[Iter: 5.2k / lr: 1.00e-4] 	Time: 57.66 (Data: 49.61) 	Err: 7.343382	
[Iter: 5.3k / lr: 1.00e-4] 	Time: 57.88 (Data: 49.75) 	Err: 7.346845	
[Iter: 5.4k / lr: 1.00e-4] 	Time: 57.71 (Data: 49.41) 	Err: 7.537878	
[Iter: 5.5k / lr: 1.00e-4] 	Time: 57.82 (Data: 49.71) 	Err: 7.401683	
[Iter: 5.6k / lr: 1.00e-4] 	Time: 58.17 (Data: 50.21) 	Err: 7.207794	
[Iter: 5.7k / lr: 1.00e-4] 	Time: 57.57 (Data: 49.46) 	Err: 7.378472	
[Iter: 5.8k / lr: 1.00e-4] 	Time: 57.74 (Data: 49.43) 	Err: 7.457488	
[Iter: 5.9k / lr: 1.00e-4] 	Time: 58.11 (Data: 50.07) 	Err: 7.313913	
[Iter: 6.0k / lr: 1.00e-4] 	Time: 58.28 (Data: 50.04) 	Err: 7.198203	
[Epoch 6 (iter/epoch: 1000)] Test time: 23.48	
(scale 4) Average PSNR: 28.2877 (Highest ever: 28.2877 at epoch = 6)	
	
[Iter: 6.1k / lr: 1.00e-4] 	Time: 59.09 (Data: 50.69) 	Err: 7.453156	
[Iter: 6.2k / lr: 1.00e-4] 	Time: 57.47 (Data: 49.34) 	Err: 7.176994	
[Iter: 6.3k / lr: 1.00e-4] 	Time: 58.58 (Data: 50.55) 	Err: 7.083494	
[Iter: 6.4k / lr: 1.00e-4] 	Time: 58.84 (Data: 50.34) 	Err: 7.062294	
[Iter: 6.5k / lr: 1.00e-4] 	Time: 57.90 (Data: 49.71) 	Err: 7.171743	
[Iter: 6.6k / lr: 1.00e-4] 	Time: 57.92 (Data: 49.89) 	Err: 7.110619	
[Iter: 6.7k / lr: 1.00e-4] 	Time: 57.22 (Data: 49.05) 	Err: 7.182309	
[Iter: 6.8k / lr: 1.00e-4] 	Time: 58.61 (Data: 50.15) 	Err: 7.099355	
[Iter: 6.9k / lr: 1.00e-4] 	Time: 58.09 (Data: 49.89) 	Err: 7.436261	
[Iter: 7.0k / lr: 1.00e-4] 	Time: 58.57 (Data: 50.47) 	Err: 7.007415	
[Epoch 7 (iter/epoch: 1000)] Test time: 23.65	
(scale 4) Average PSNR: 28.3610 (Highest ever: 28.3610 at epoch = 7)	
	
[Iter: 7.1k / lr: 1.00e-4] 	Time: 59.49 (Data: 50.68) 	Err: 7.013370	
[Iter: 7.2k / lr: 1.00e-4] 	Time: 58.05 (Data: 49.62) 	Err: 7.256025	
[Iter: 7.3k / lr: 1.00e-4] 	Time: 57.96 (Data: 49.75) 	Err: 7.065247	
[Iter: 7.4k / lr: 1.00e-4] 	Time: 58.43 (Data: 50.19) 	Err: 7.236178	
[Iter: 7.5k / lr: 1.00e-4] 	Time: 57.86 (Data: 49.75) 	Err: 7.288727	
[Iter: 7.6k / lr: 1.00e-4] 	Time: 58.21 (Data: 50.14) 	Err: 6.995226	
[Iter: 7.7k / lr: 1.00e-4] 	Time: 58.45 (Data: 50.10) 	Err: 7.003108	
[Iter: 7.8k / lr: 1.00e-4] 	Time: 58.34 (Data: 50.10) 	Err: 6.968335	
[Iter: 7.9k / lr: 1.00e-4] 	Time: 58.06 (Data: 49.99) 	Err: 6.904951	
[Iter: 8.0k / lr: 1.00e-4] 	Time: 58.26 (Data: 49.67) 	Err: 7.103047	
[Epoch 8 (iter/epoch: 1000)] Test time: 23.62	
(scale 4) Average PSNR: 28.4066 (Highest ever: 28.4066 at epoch = 8)	
	
[Iter: 8.1k / lr: 1.00e-4] 	Time: 59.04 (Data: 50.64) 	Err: 7.247079	
[Iter: 8.2k / lr: 1.00e-4] 	Time: 58.20 (Data: 49.88) 	Err: 7.180099	
[Iter: 8.3k / lr: 1.00e-4] 	Time: 58.24 (Data: 49.67) 	Err: 6.996955	
[Iter: 8.4k / lr: 1.00e-4] 	Time: 58.86 (Data: 50.49) 	Err: 7.076885	
[Iter: 8.5k / lr: 1.00e-4] 	Time: 58.23 (Data: 50.02) 	Err: 7.028236	
[Iter: 8.6k / lr: 1.00e-4] 	Time: 57.68 (Data: 49.56) 	Err: 7.116949	
[Iter: 8.7k / lr: 1.00e-4] 	Time: 57.59 (Data: 49.54) 	Err: 7.182252	
[Iter: 8.8k / lr: 1.00e-4] 	Time: 57.99 (Data: 49.85) 	Err: 7.182454	
[Iter: 8.9k / lr: 1.00e-4] 	Time: 57.79 (Data: 49.45) 	Err: 7.205822	
[Iter: 9.0k / lr: 1.00e-4] 	Time: 57.88 (Data: 49.67) 	Err: 7.369923	
[Epoch 9 (iter/epoch: 1000)] Test time: 23.59	
(scale 4) Average PSNR: 28.5147 (Highest ever: 28.5147 at epoch = 9)	
	
[Iter: 9.1k / lr: 1.00e-4] 	Time: 58.90 (Data: 50.38) 	Err: 7.108896	
[Iter: 9.2k / lr: 1.00e-4] 	Time: 58.35 (Data: 50.30) 	Err: 7.120156	
[Iter: 9.3k / lr: 1.00e-4] 	Time: 57.81 (Data: 49.63) 	Err: 6.699856	
[Iter: 9.4k / lr: 1.00e-4] 	Time: 57.45 (Data: 49.53) 	Err: 7.055263	
[Iter: 9.5k / lr: 1.00e-4] 	Time: 58.29 (Data: 50.09) 	Err: 7.094778	
[Iter: 9.6k / lr: 1.00e-4] 	Time: 58.64 (Data: 50.43) 	Err: 7.121905	
[Iter: 9.7k / lr: 1.00e-4] 	Time: 59.03 (Data: 50.64) 	Err: 6.954122	
[Iter: 9.8k / lr: 1.00e-4] 	Time: 64.40 (Data: 56.23) 	Err: 7.131248	
Warning: Error is too large! Skip this batch. (Err: 15.775752)	
[Iter: 9.9k / lr: 1.00e-4] 	Time: 69.52 (Data: 61.19) 	Err: 7.178918	
[Iter: 10.0k / lr: 1.00e-4] 	Time: 72.00 (Data: 63.93) 	Err: 6.885784	
[Epoch 10 (iter/epoch: 1000)] Test time: 23.70	
(scale 4) Average PSNR: 28.5899 (Highest ever: 28.5899 at epoch = 10)	
	
Warning: Error is too large! Skip this batch. (Err: 14.525774)	
[Iter: 10.1k / lr: 1.00e-4] 	Time: 69.08 (Data: 60.68) 	Err: 6.986968	
[Iter: 10.2k / lr: 1.00e-4] 	Time: 72.27 (Data: 64.09) 	Err: 7.137931	
[Iter: 10.3k / lr: 1.00e-4] 	Time: 72.05 (Data: 63.93) 	Err: 7.256474	
[Iter: 10.4k / lr: 1.00e-4] 	Time: 74.87 (Data: 66.77) 	Err: 6.899007	
[Iter: 10.5k / lr: 1.00e-4] 	Time: 73.01 (Data: 64.82) 	Err: 6.969793	
[Iter: 10.6k / lr: 1.00e-4] 	Time: 75.41 (Data: 67.26) 	Err: 6.985296	
[Iter: 10.7k / lr: 1.00e-4] 	Time: 99.23 (Data: 91.19) 	Err: 7.097432	
[Iter: 10.8k / lr: 1.00e-4] 	Time: 121.62 (Data: 113.34) 	Err: 6.992342	
[Iter: 10.9k / lr: 1.00e-4] 	Time: 142.16 (Data: 133.66) 	Err: 7.034576	
[Iter: 11.0k / lr: 1.00e-4] 	Time: 169.72 (Data: 161.17) 	Err: 7.181707	
[Epoch 11 (iter/epoch: 1000)] Test time: 24.06	
(scale 4) Average PSNR: 28.6195 (Highest ever: 28.6195 at epoch = 11)	
	
[Iter: 11.1k / lr: 1.00e-4] 	Time: 219.76 (Data: 211.10) 	Err: 7.121071	
[Iter: 11.2k / lr: 1.00e-4] 	Time: 231.93 (Data: 223.22) 	Err: 6.955821	
[Iter: 11.3k / lr: 1.00e-4] 	Time: 233.64 (Data: 224.90) 	Err: 6.994724	
Killed

the result on BSD100

Hi~
I want to know why the PSNR and SSIM values on BSD100 when I run the pre-trained code are worse than your published a lot . Thank you.

SR images of SRResnet

Thank you for your wonderful work. We all know that SRGAN and SRResnet do not provide any code. For the SR image of the test set, I can only find the SR images of set5 and set14. But there are other SR images of the test set in your paper (BSD100, Urban100). I also need these SR images. How do you get these images?

Asking for the model(challenge&paper)(Links seem to be broken)

It sames that the two links1 and links2 of downloading trained models given in the repo have broken down, I can't download them. Could anybody provide me a backup version?

Thanks a lot :).

Edit: And if possible, could you please also provide the DIV2K datasets download links, i can't download it too.
Thank you very much!

Segmentation fault when training baseline model

I got message below when training baseline model:

...
[Iter: 299.1k / lr: 5.00e-5] Time: 66.29 (Data: 61.42) Err: 3.234126
[Iter: 299.2k / lr: 5.00e-5] Time: 65.32 (Data: 60.11) Err: 3.496183
[Iter: 299.3k / lr: 5.00e-5] Time: 66.40 (Data: 61.23) Err: 3.399313
[Iter: 299.4k / lr: 5.00e-5] Time: 64.99 (Data: 60.01) Err: 3.379927
[Iter: 299.5k / lr: 5.00e-5] Time: 65.95 (Data: 60.72) Err: 3.503887
[Iter: 299.6k / lr: 5.00e-5] Time: 66.23 (Data: 61.05) Err: 3.338660
[Iter: 299.7k / lr: 5.00e-5] Time: 65.30 (Data: 59.97) Err: 3.448611
[Iter: 299.8k / lr: 5.00e-5] Time: 65.69 (Data: 60.95) Err: 3.330575
[Iter: 299.9k / lr: 5.00e-5] Time: 66.04 (Data: 61.20) Err: 3.350167
[Iter: 300.0k / lr: 5.00e-5] Time: 65.34 (Data: 59.59) Err: 3.413485
[Epoch 300 (iter/epoch: 1000)] Test time: 25.48
(scale 2) Average PSNR: 35.5833 (Highest ever: 35.5902 at epoch = 288)

Segmentation fault (core dumped)

I'm not sure it the training process is successfully completed or not. If it is, where is the trained model?

which forward is better?

there are 3 forward function defined in code/utils.lua

chopFoward is much faster then recursiveForward

it takes 30 secs to go through recursiveForward.
it only take 1 sec to go through chopFoward.

so does it means the quality of recursiveForward is better than chopForward?

Preprocessing in this work

In the paper, you mentioned the only preprocessing you did was subtracting the mean RGB value from entire dataset. But did you also preprocess your label(high resolution image) in the same way, subtracting the mean RGB value from all the label images? Another thing I noticed was the constant 255 was used to scale the input in you code. Is it because the dynamic range of input is [0,1]?

Could you show more details about your image preprocessing?

Thanks,

Recurrence the model using pytorch

Hi,
I am trying to recurrence baseline model x2 in pytorch. I think I create my model totally the same to the paper and baseline.lua.
I convert your model baseline.x2.t7 to baseline.x2.pth and found that the parameters are different to my pytorch model's.
Did you add 2 more conv() in your training?Or is there someting wrong in my steps?

my baseline params	size	baseline origin params	size
conv_in.weight	64x3x3x3	0.weight	64x3x3x3
xxx	xxx	1.weight	64x64x3x3
residual.layer.0.conv1.weight	64x64x3x3	2.0.0.0.0.0.0.weight	64x64x3x3
residual.layer.0.conv2.weight	64x64x3x3	2.0.0.0.0.0.2.weight	64x64x3x3
esidual.layer.1.conv1.weight	64x64x3x3	2.0.0.1.0.0.0.weight	64x64x3x3
residual.layer.1.conv2.weight	64x64x3x3	2.0.0.1.0.0.2.weight	64x64x3x3
……	……	……	……
residual.layer.15.conv2.weight	64x64x3x3	2.0.0.15.0.0.2.weight	64x64x3x3
conv_mid.weight	64x64x3x3	2.0.0.16.weight	64x64x3x3
conv3.weight	256x64x3x3	3.0.weight	256x64x3x3
conv_out.weight	3x64x3x3	4.weight	3x64x3x3
xxx	xxx	5.weight	3x3x1x1

What's the license for tool codes in this repo, can I use and modify?

Does the use of Flickr2k dataset obey the CC-BY 4.0 license?

Hello, thank you for sharing the codes and datasets. I plan to use the flickr2k dataset in a CVF open-source paper, thus I would like to know the term of use of this dataset, whether it obey the CC-BY 4.0 license, and is it ok to be used in the CVF open-source papers.

where can I get Flickr2K dataset?

Hi, LimBee,
I want to get Flickr2K dataset but I can not find this dataset now.

Pixel mean substraction

Hi,

I was wondering how did you compute the dataset mean that you substract/add at the start/end of the training. Did you add all red/green/blue components in all pixels, in all images?

Cannot test for scale x3 and x4, utils.lua:313: bad argument #1 to 'copy' (sizes do not match

Dear all,
I have encountered the issue that I cannot test for scale x3 and x4. But it's ok for scale x2.
The error message is as follows:
th test.lua -model EDSR_x3 -selfEnsemble false
Preparing dataset...
[Load time] 0.035s (average 0.017s)

Model: [EDSR_x3]

[1/2] 0901x3.png /home/yulun/torch/install/bin/luajit: .../NTIRE2017/code/utils.lua:313: bad argument #1 to 'copy' (sizes do not match at /home/yulun/torch/extra/cutorch/lib/THC/THCTensorCopy.cu:31)
stack traceback:
[C]: in function 'copy'
...yulun/Disk6T/papers/CVPR17EDSR/NTIRE2017/code/utils.lua:313: in function 'chopForward'
...yulun/Disk6T/papers/CVPR17EDSR/NTIRE2017/code/utils.lua:320: in function 'chopForward'
test.lua:235: in main chunk
[C]: in function 'dofile'
...ulun/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406620

result

Due to time, I have no time to run the experiment of EDSR, so could you please send me the result of the test with div2k? I want to compare the results of my experiment when I write my paper。

What's the network architecture of the downsampler? How to train it?

cannot load incompatible bytecodeWarning

Hello..

I download the code and run on my own image. The code and produce the result but i got one warning. A little bit worry about it if the proper code not load than may the result is not ok. The screen shot is attached. Please guide me.

About Training Time and Gpu

Hi LimBee, I already learned that you have been trained several days. Can you tell me which type of Gpu are you using and how many of them? Cause I have only one single Nvidia 1080.

Discrepancy

Is there any discrepancies of PSNR using DIV2K vs Flickr2K vs DIV2K&Flickr2K as training set?

Combine 800 train and 100 val images into the 'train' folder

Dear, all

To run the code, we should make some modifications about the original provided DIV2K dataset.
Copy 100 images in DIV2K_valid_HR into DIV2K_train_HR.
Copy 100 images in DIV2K_valid_LR_bicubic(unknown)/X2(X3/X4) into DIV2K_train_LR_bicubic(unknown)/X2(X3/X4).

Can I use my dataset for training?

Thanks for you work, but for the DIV2K dataset, I just reduce the number of images from 900 to 10 in every folders, there is a problem occur during training time, the error as following:

loading model and criterion...
Creating model from file: models/baseline.lua
Creating data loader...
loading data...
Initializing data loader for train set...
Initializing data loader for val set...
Train start
/home/hao/torch/install/bin/luajit: /home/hao/torch/install/share/lua/5.1/threads/threads.lua:183: [thread 1 callback] /home/hao/torch/install/share/lua/5.1/image/init.lua:367: /home/hao/NTIRE2017/dataset/DIV2K/DIV2K_train_LR_bicubic/X2/0403x2.png: No such file or directory
stack traceback:
[C]: in function 'error'
/home/hao/torch/install/share/lua/5.1/image/init.lua:367: in function 'load'
./data/div2k.lua:122: in function 'get'
./dataloader.lua:89: in function <./dataloader.lua:76>
[C]: in function 'xpcall'
/home/hao/torch/install/share/lua/5.1/threads/threads.lua:234: in function 'callback'
/home/hao/torch/install/share/lua/5.1/threads/queue.lua:65: in function </home/hao/torch/install/share/lua/5.1/threads/queue.lua:41>
[C]: in function 'pcall'
/home/hao/torch/install/share/lua/5.1/threads/queue.lua:40: in function 'dojob'
[string " local Queue = require 'threads.queue'..."]:15: in main chunk
stack traceback:
[C]: in function 'error'
/home/hao/torch/install/share/lua/5.1/threads/threads.lua:183: in function 'dojob'
./dataloader.lua:158: in function '(for generator)'
./train.lua:69: in function 'train'
main.lua:33: in main chunk
[C]: in function 'dofile'
.../hao/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670

but when I use the entire images, it works.
So, the questions are, (i) should I need to use the whole images in DIV2K to train the model and (ii) may I use my dataset for training and testing?
Thanks a lot!

Issues when training with my own image

Hi, @limbee , Thank you for sharing your work. I have a problem need you help, thank you!
I use part images of DIV2K dataset. In every folders of DIV2K dataset, i choose 30 images, and the number is 0001-0030. I occur a error when training with these choose images, error information as follows:
loading model and criterion...
Creating model from file: models/baseline.lua
Creating data loader...
loading data...
Initializing data loader for train set...
Initializing data loader for val set...
Train start
[Iter: 0.1k / lr: 1.00e-4] Time: 98.73 (Data: 93.01) Err: 13.301673
[Iter: 0.2k / lr: 1.00e-4] Time: 96.38 (Data: 91.71) Err: 7.238939
[Iter: 0.3k / lr: 1.00e-4] Time: 96.18 (Data: 91.42) Err: 6.127012
[Iter: 0.4k / lr: 1.00e-4] Time: 100.27 (Data: 95.47) Err: 5.501482
[Iter: 0.5k / lr: 1.00e-4] Time: 95.15 (Data: 90.37) Err: 5.155260
[Iter: 0.6k / lr: 1.00e-4] Time: 97.84 (Data: 93.15) Err: 5.166865
[Iter: 0.7k / lr: 1.00e-4] Time: 97.32 (Data: 92.40) Err: 5.002103
[Iter: 0.8k / lr: 1.00e-4] Time: 98.88 (Data: 94.14) Err: 4.949536
[Iter: 0.9k / lr: 1.00e-4] Time: 98.07 (Data: 93.16) Err: 4.653362
[Iter: 1.0k / lr: 1.00e-4] Time: 98.70 (Data: 93.79) Err: 4.809450
/home/halo_op/torch/install/bin/luajit: .../halo_op/torch/install/share/lua/5.1/threads/threads.lua:183: [thread 1 callback] /home/halo_op/torch/install/share/lua/5.1/image/init.lua:367: /data/csfu/dataset/DIV2K_part/DIV2K/DIV2K_train_LR_bicubic/X2/0801x2.png: No such file or directory
stack traceback:
[C]: in function 'error'
/home/halo_op/torch/install/share/lua/5.1/image/init.lua:367: in function 'load'
./data/div2k.lua:122: in function 'get'

ps: I use the command begin training as follows:
th main.lua -datadir /data/dataset/DIV2K_part -nTrain_DIV2K 30 -datatype png

Looking forward to your reply, thank you!

torch.*Tensor expected, got table

Hello,

I am running the following command:

[ec2-user@ demo]$ th test.lua -type test -dataset myData -model MDSR -scale 2 -degrade [bicubic]

I am getting the following error:

/home/ec2-user/src/torch/install/bin/luajit: /rick/ntire2017/code/utils.lua:315: bad argument #1 to 'copy' (torch.*Tensor expected, got table)
stack traceback:
[C]: in function 'copy'
/rick/ntire2017/code/utils.lua:315: in function 'chopForward'
test.lua:270: in main chunk
[C]: in function 'dofile'
.../src/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004064c0

The section of code that errors is (in chopForward method in utils.lua)
outputPatch[{{i, i + nGPU - 1}, {}, {}, {}}]:copy(model:forward(inputBatch))

And if I run this command

th test.lua -type test -dataset myData -model EDSR -scale 2 -degrade bicubic

I get a different error at the same section of code

/home/ec2-user/src/torch/install/bin/luajit: /rick/ntire2017/code/utils.lua:315: bad argument #1 to 'copy' (sizes do not match at /home/ec2-user/src/torch/extra/cutorch/lib/THC/THCTensorCopy.cu:31)
stack traceback:
[C]: in function 'copy'
/rick/ntire2017/code/utils.lua:315: in function 'chopForward'
test.lua:270: in main chunk
[C]: in function 'dofile'
.../src/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004064c0

Any help is greatly appreciated. I am running on the Machine Learning AMI on amazon ec2.

problem about training own dataset

hello, I find some problems when I train the model with my own dataset. Below are some description about my issue, hoping you can give me some suggestion about this problem, thank you very much !
2019-02-20_20-27-28

loading model and criterion...
Loading pre-trained model from: ../demo/model/EDSR_x4.t7
Creating data loader...
loading data...
Initializing data loader for train set...
Initializing data loader for val set...
Train start
/home/luomeilu/torch/install/bin/luajit: ...luomeilu/torch/install/share/lua/5.1/threads/threads.lua:183: [thread 1 callback] /home/luomeilu/torch/install/share/lua/5.1/image/init.lua:367: /var/tmp/dataset/DIV2K/DIV2K_train_LR_bicubic/X4/0045x4.png: No such file or directory
stack traceback:
[C]: in function 'error'
/home/luomeilu/torch/install/share/lua/5.1/image/init.lua:367: in function 'load'
./data/div2k.lua:122: in function 'get'
./dataloader.lua:89: in function <./dataloader.lua:76>
[C]: in function 'xpcall'
...luomeilu/torch/install/share/lua/5.1/threads/threads.lua:234: in function 'callback'
...e/luomeilu/torch/install/share/lua/5.1/threads/queue.lua:65: in function <...e/luomeilu/torch/install/share/lua/5.1/threads/queue.lua:41>
[C]: in function 'pcall'
...e/luomeilu/torch/install/share/lua/5.1/threads/queue.lua:40: in function 'dojob'
[string " local Queue = require 'threads.queue'..."]:15: in main chunk
stack traceback:
[C]: in function 'error'
...luomeilu/torch/install/share/lua/5.1/threads/threads.lua:183: in function 'dojob'
./dataloader.lua:158: in function '(for generator)'
./train.lua:69: in function 'train'
main.lua:33: in main chunk
[C]: in function 'dofile'
...eilu/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670

What is the input channel in the Set5 / Set14 test in the paper? RGB or Y?

What is the input channel in the Set5 / Set14 test in the paper? RGB or Y? Thanks for your answer.

where are the all models

HI, i'm very interested in your works.
I want to try your models for some little project.
But look into your models dir, there aren't any model.
Can you provide the trained models ?
Thanks :)

Error when converting DIV2K dataset to t7 files

I've placed the DIV2K dataset in a custom location, changed the cmd to convert png -> t7 accordingly but before any conversion happens i get this error:

th png_to_t7.lua -apath /mycustomfolder/div2k -dataset DIV2K

Converting /mycustomfolder/div2k/DIV2K_train_HR
/Users/me/torch/install/bin/luajit: png_to_t7.lua:113: bad argument #1 to 'sort' (table expected, got nil)
stack traceback:
	[C]: in function 'sort'
	png_to_t7.lua:113: in main chunk
	[C]: in function 'dofile'
	...me/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x0100b54360

I've got the following folders of the DIV2K dataset:

DIV2K_train_HR
DIV2K_train_LR_bicubic
DIV2K_train_LR_unknown
DIV2K_valid_HR
DIV2K_valid_LR_bicubic

PS: The link to your server isn't working anymore: http://cv.snu.ac.kr/research/EDSR/DIV2K.tar
Could you please explain how to download the images from https://competitions.codalab.org/competitions/16303? There is no direct download link or data site.

Number of training patches

Hi,

Please can you confirm how many patches are generated from 800 DIV2K train images .

I mean the stride you use for extracting patches.

One more thing, in one of your presentation, you are saying that the size is 96x96 while in paper you are using 48x48. Please can you confirm which is the correct size you use.

why does the inference so slow

hi , i'm using the test.lua script in demo to inference a image.
the model i used is EDSR_x2.t7
the gpu i used is Tesla K80
the image i used is 180x100

i type the following command to run the script:
th test.lua -type test -model EDSR_x2.t7 -dataset img_input -gpuid 3

it tooks about 30 seconds to inference
why is it so slow, is there any problem ?

Can I combine it with GAN?

Hello,LimBee,I just read your paper,the result is really cool,and EDSR gets the highest score in PSNR,but it seems it still has a sense of blurr in vision.I noted that you changed the loss function from l2 to l1,(though I don't know why it's better) As the "SRGAN" paper said:With a high PSNR,the results are often over smoothed,lack high-frequency detail,making them perceptually unsatisfying. And I want to change the l1 loss into perceptually loss, (combine EDSR with GAN),or using EDSR instead of SRResNet in SRGAN,will it be better ?

GPU RAM required for 'test'

Greetings. Thanks for writing this paper and making your code repo available.

I've been attempting to reproduce the DIV2K results. I was able to generate t7 files without issue, and am able to run the 'train' code for the X2 bilinear model (training from scratch).

However, after the first 1000 iterations or so, when the code attempts to do a "test" cycle, it runs out of GPU ram. How much GPU ram is required to run the "test" section? Is there any way to reduce the RAM requirements for "test"? I tried dropping the mini-batch size but this only seems to affect RAM usage in "train".

For now I've disabled the "test" section and am creating a dummy PSNR array to enable training to run, but this makes it impossible to track PSNR during training (all I can see is loss going down).

paper

Excuse me, I am very interested in your code but where is your paper? I didn't see it in this Repo.

About Discriminator Network

Did you modify the Discriminator Network of SRResNet ! Thanks!

MDSR not working

Hi, I am having some trouble running the demos. if I run

th test.lua -model EDSR_x2 -selfEnsemble false

it works fine, but then when I run

th test.lua -model MDSR -scale 2 -selfEnsemble false

I get this error:

Model: [MDSR]
This is a multi-scale model! Swap the model
/opt/torch/install/bin/luajit: test.lua:64: attempt to index a nil value
stack traceback:
	test.lua:64: in function 'swap'
	test.lua:222: in main chunk
	[C]: in function 'dofile'
	/opt/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x00406670

or if I run

th test.lua -model EDSR_x4 -selfEnsemble false

I get this error:

Model: [EDSR_x4]
>> [1/3]	2276523950_e06e9065b2_z.jpg	/opt/torch/install/bin/luajit: /NTIRE2017/code/utils.lua:313: bad argument #1 to 'copy' (sizes do not match at /opt/torch/extra/cutorch/lib/THC/THCTensorCopy.cu:31)
stack traceback:
	[C]: in function 'copy'
	/NTIRE2017/code/utils.lua:313: in function 'chopForward'
	/NTIRE2017/code/utils.lua:320: in function 'chopForward'
	test.lua:252: in main chunk
	[C]: in function 'dofile'
	/opt/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x00406670

Any idea what I am doing wrong? Thanks!