hengchuan/rdn-tensorflow

A TensorFlow implementation of CVPR 2018 paper "Residual Dense Network for Image Super-Resolution".

rdn-tensorflow's Issues

Set5 test set evaluation result

Hi,
I used the model you pretrained with 200 epochs and evaluated the model on Set5 test set. But I only got ~25.5 average PSNR which is somewhat poor. Anybody same with me? Should I train the model continuely ?

ERROR with checkpoint

Hi! I'm learning the RDN. Luckly,I find your works.thanks!
but I meet some troubles,when I run it ,The problem is following :OSError: Unable to open file (unable to open file: name = './checkpoint\train.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0).and I have load the DIV2K ! some idea.Thank you!

img = img[0:h, 0:w, :] TypeError: slice indices must be integers or None or have an __index__ method

Did u use Gaussian kernel to blur HR image?

loss keep stable after first several batch?

loss never go down after many batch, after many epoch loss still between 0.1 and 0.2, any help?

模型

你好，感谢你的代码，请问有已经训练好的模型吗？

the model was trained on DIV2k? test SET5 on it , but the psnrs are 17~30.

the model was trained on DIV2k? test SET5 on it , but the psnrs are 17~30。 Can it be reach the same result of the paper ?

Is there something wrong with the network？

I download the code, and run testing using pre-trained model in Windows:
python main.py --is_train=False --is_eval=False --test_img=Test/Set5/butterfly.png

but an error occurs at model.py in function "def test(self, config): ". At line :
result = self.sess.run([self.pred], feed_dict={self.images: input_ / 255.0})

error information as follows:
Inputs to operation ArithmeticOptimizer/AddOpsRewrite_Leaf_1_Add_11 of type _MklAddN must have the same size and shape. Input 0: [4194304] != input 1: [1,256,256,64]
[[Node: ArithmeticOptimizer/AddOpsRewrite_Leaf_1_Add_11 = _MklAddN[N=2, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](Conv2D_10, add_1, Conv2D_10:2, DMT/_324)]]

The shape of Input Image is (1, 256, 256, 3)，so I think there maybe be some wrong in somewhere in the network.

about trainset

the pretrained_model in chechpoint is trained only by DIV2K training data? or there is another trainset?
when I train a new model(scale=2), the performance is not as good as the chekpointmodel(scale=3) you offered, so is there dataset problem ? or it should be less performance?

这个地方为什么要逆序呢

Error with checkpoint during training

Hi!
I want to train the model using x2 resolution, but when I run: python main.py --scale=2
I get:
IOError: Unable to open file (unable to open file: name = './checkpoint/train.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

That is true, I don't have any checkpoint beacouse I can't train the model.

Some idea?

Thanks

the psnr is not good.

I use your checkpoint to predict PSNR on the set14, some images are not good.

Phase shift concatenation fails with --scale 1

I am trying to adapt the network for other applications than super-resolution, using --scale 1. However, in model._phase_shift(), I get the following error, on the concatenation:

  File "main.py", line 53, in <module>
    tf.app.run()
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "main.py", line 45, in main
    rdn.train(FLAGS)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 278, in train
    self.build_model(images_shape, labels_shape)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 190, in build_model
    self.pred = self.model()
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 233, in model
    FU = self.UPN(FDF)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 202, in UPN
    x = self.PS(x, self.scale)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 265, in PS
    X = tf.concat([self._phase_shift(x, r) for x in Xc], 3) # Do the concat RGB
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 265, in <listcomp>
    X = tf.concat([self._phase_shift(x, r) for x in Xc], 3) # Do the concat RGB
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 246, in _phase_shift
    X = tf.concat([tf.squeeze(x) for x in X], 2)  # bsize, b, a*r, r
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1124, in concat
    return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1033, in concat_v2
    "ConcatV2", values=values, axis=axis, name=name)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in __init__
    control_input_ops)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be at least rank 3 but is rank 2 for 'concat_16' (op: 'ConcatV2') with input shapes: [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [] and with computed input tensors: input[32] = <2>.

From my research it seems to be due to attempting to concatenate over a non-existent dimension. However, I am not familiar enough with TensorFlow, RDN, or your implementation to debug this and could not find a reference to phase shifting in the original paper and implementation of RDN, so if --scale 1 is a special case that must be handled separately, please advise on how to make the necessary changes.

I confirmed that --scale 3 works correctly in my environment and restarted from the preprocessing phase to ensure that my data is correct.

AttributeError: 'NoneType' object has no attribute 'shape'

你好，为什么我运行更新后的代码会出现这个问题呢？求教

How could I have a look at the test result ?And what should I do if I want to test my own pictures?

PIL和MatLAB的resize 效果一致？

我看了您的代码，写的非常好，我打算使用您的代码。我知道CV2和Matlab的Resize是不同的，那么PIL和MatLab得resize效果等同吗？
您的预训练模型是基于MatLab resize，测试的时候使用PIL resize 效果是不是一致呢？

run in python3

hi @hengchuan
thanks for sharing your work with us
i use windows, Unfortunately I can not use this project in python 2.7, any chance to get it work with python 3.5 please.

COLOR_BGR2YCR_CB

t = cv2.cvtColor(sub_label, cv2.COLOR_BGR2YCR_CB)
请问utils.py里 BGR到YCrCb色彩空间转换之后这段代码有什么作用呢？
t = t[:, :, 0]
gx = t[1:, 0:-1] - t[0:-1, 0:-1]
gy = t[0:-1, 1:] - t[0:-1, 0:-1]
Gxy = (gx2 + gy2)**0.5
r_gxy = float((Gxy > 10).sum()) / ((config.image_size*config.scale)**2) * 100
if r_gxy < 10:
continue

indices error while training

Hi. I encountered with the following error while training:
`Prepare Data...

Traceback (most recent call last):
File "main.py", line 53, in
tf.app.run()
File "/home/hermes/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 124, in run
sys.exit(main(argv))
File "main.py", line 45, in main
rdn.train(FLAGS)
File "/home/hermes/RDN-TensorFlow-master/model.py", line 272, in train
input_setup(config)
File "/home/hermes/RDN-TensorFlow-master/utils.py", line 194, in input_setup
make_sub_data(data, config)
File "/home/hermes/RDN-TensorFlow-master/utils.py", line 126, in make_sub_data
input, label_, = preprocess(data[i], config.scale, eng, mdouble)
File "/home/hermes/RDN-TensorFlow-master/utils.py", line 55, in preprocess
label_ = modcrop(img, scale)
File "/home/hermes/RDN-TensorFlow-master/utils.py", line 45, in modcrop
img = img[0:h, 0:w, :]
TypeError: slice indices must be integers or None or have an index method`

I used python3 branch. Also I should mention that I put all images to RDN-TensorFlow/Train/DIV2K_train_HR.
What should I do now?
Thanks

Validating while training?

Hello, I want to ask if you don't need to train and validate when you do the experiment? So how do you find the best model?

Error with SystemExit：exception :no description when the training is complete in tf.app.run

The problem is following:
when the training or test is complete, and all the training data is displayed and the model is saved.However,
The Error with SystemExit was appear in tf.app.run(),The error message likes this: SystemExit：exception :no description.
I desire your nice ideas,Thenk you!

Error of image shape when testing

I have got this error :

2018-09-18 14:52:42.750157: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 7.17GiB
2018-09-18 14:52:42.750157: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)

Prepare Data...

['C:\Users\someone\eclipse-workspace\k_tn\Test\Set5\butterfly_GT.bmp']

Now Start Testing...

(1, 85, 85, 3)
(1, 256, 256, 3)
(1, 85, 85, 27)
(1, 255, 255, 3)
Traceback (most recent call last):
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
input_tensors_as_shapes, status)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 256 and 255 for 'sub' (op: 'Sub') with input shapes: [1,256,256,3], [1,255,255,3].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\someone\eclipse-workspace\k_tn\main.py", line 46, in
tf.app.run()
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run
_sys.exit(main(argv))
File "C:\Users\someone\eclipse-workspace\k_tn\main.py", line 43, in main
rdn.test(FLAGS)
File "C:\Users\someone\eclipse-workspace\k_tn\model.py", line 334, in test
self.build_model(images_shape, labels_shape)
File "C:\Users\someone\eclipse-workspace\k_tn\model.py", line 192, in build_model
self.loss = tf.reduce_mean(tf.square(self.labels - self.pred))
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\math_ops.py", line 907, in binary_op_wrapper
return func(x, y, name=name)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 5424, in _sub
"Sub", x=x, y=y, name=name)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 3162, in create_op
compute_device=compute_device)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 3208, in _create_op_helper
set_shapes_for_outputs(op)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2427, in set_shapes_for_outputs
return _set_shapes_for_outputs(op)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2400, in _set_shapes_for_outputs
shapes = shape_func(op)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2330, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "C:\Users\someone\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Dimensions must be equal, but are 256 and 255 for 'sub' (op: 'Sub') with input shapes: [1,256,256,3], [1,255,255,3].

psnr

Why do you use your checkpoint to test the PSNR of set5 and set14? The PSNR of set5 is only 31.5
and 29.0

Is this code for image denoising ?

Performance issues in model.py(P2)

Hello,I found a performance issue in the definition of eval ,
model.py,
self.sess = tf.Session() was repeatedly called in for idx in range(data_num): and was not closed.
I think it will increase the efficiency and avoid out of memory if you close this session after using it.

The same issues also exist in self.sess = tf.Session()

Here are two files to support this issue,support1 and support2

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

Training for Scale 3 over pretrained model of Scale 2

Hi,
Could you please let me know how to train a model for scale 3, by using the pretrained model of scale 2. Right now I'm getting the following error , if I try to train for scale 3, using checkpoints from scale 2:
InvalidArgumentError : Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint

I'm able to train for scale 2 and scale 3 separately, but am unable to train for scale 3 over already trained scale 2.

没有train.h5文件？

您好，十分感谢您公开代码，我在学习的时候遇到下面这个问题，找不到train.h5文件，请问这是在哪里下载呢，蟹蟹

RND+？

the self-ensemble was not put into practice? thanks

Size of the Scale factor

Hi. Is there any way to change size of the scale factor? What should I do if I want outputs with x4 or x8 scaling?
When I change the scale factor from 3 to 4, I receive this error:
ValueError: Cannot feed value of shape (128, 96, 96, 3) for Tensor 'labels:0', which has shape '(?, 128, 128, 3)
Thanks.

Loss Function

Hi, thanks for your tensorflow implement .
I find that you use L2 loss in the model.py, it's different from the paper. Why we not use the L1 loss, can L2 loss bring some benefit for training ?

An error about h5py cannot be solved

input_ = hf['input']
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-tnf92dft-build/h5py/_objects.c:2853)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-tnf92dft-build/h5py/_objects.c:2811)
File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/group.py", line 166, in getitem
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-tnf92dft-build/h5py/_objects.c:2853)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-tnf92dft-build/h5py/_objects.c:2811)
File "h5py/h5o.pyx", line 190, in h5py.h5o.open (/tmp/pip-tnf92dft-build/h5py/h5o.c:3739)
KeyError: "Unable to open object (Object 'input' doesn't exist)"

What should I do?

Black output

Training this model "as it comes" on the suggested data set and testing it on the test image leads to completely black pictures.

Training with
python main.py

testing with
python main.py --is_train=False --test_img='Test/Set5/butterfly_GT.bmp' --result_dir='Test/results'

where is the "RDN-TensorFlow/Train/DIV2K_train_HR"

您好，打扰一下，我是刚开始接触RDN这些，对于代码运行不是很熟悉，我想运行一下您的代码学习，但是没有搞明白怎么读测试数据，您说把div2k数据放到 RDN-TensorFlow/Train/DIV2K_train_HR这个是自己建立的文件夹么？就在你的项目RDN-TensorFlow-master里面依次建立那几个文件夹么