crashmoon/progressive-generative-networks

Semantic Image Inpainting with Progressive Generative Networks

Progressive-Generative-Networks

Haoran Zhang, Zhenzhen Hu, Changzhi Luo, Wangmeng Zuo, Meng Wang:

Semantic Image Inpainting with Progressive Generative Networks.

ACM Multimedia 2018: 1939-1947 https://dl.acm.org/citation.cfm?doid=3240508.3240625

@inproceedings{Zhang2018SemanticII, title={Semantic Image Inpainting with Progressive Generative Networks}, author={Haoran Zhang and Zhenzhen Hu and Changzhi Luo and Wangmeng Zuo and Meng Wang}, booktitle={ACM Multimedia}, year={2018} }

Prerequisites

Python, NumPy, SciPy, Matplotlib

A recent NVIDIA GPU

A latest master version of Pytorch

Progress

1.Make a CSV file with make_images.py

2.Modify the parameters in gan_lstm.py (Param.out_path)

3.python gan_lstm.py (run the program)

Results

progressive-generative-networks's Issues

when i run the gan_lstm.py an error happened

gan_lstm.py:89: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
nn.init.kaiming_normal(m.weight.data, mode='fan_out')
gan_lstm.py:217: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
nn.init.kaiming_normal(m.weight.data, mode='fan_out')
<main.ParisData object at 0x7f6c003cc090>
/usr/local/lib/python2.7/dist-packages/skimage/transform/_warps.py:84: UserWarning: The default mode, 'constant', will be changed to 'reflect' in skimage 0.15.
warn("The default mode, 'constant', will be changed to 'reflect' in "
/usr/local/lib/python2.7/dist-packages/skimage/transform/_warps.py:84: UserWarning: The default mode, 'constant', will be changed to 'reflect' in skimage 0.15.
warn("The default mode, 'constant', will be changed to 'reflect' in "
/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py:1332: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
Traceback (most recent call last):
File "gan_lstm.py", line 583, in
main()
File "gan_lstm.py", line 490, in main
D_cost = fake_loss_48.data[0] + fake_loss_32.data[0] + fake_loss_16.data[0] + fake_loss_0.data[0]
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

Trained model and dataset

Can you share the trained model and relevant dataset, if it is ok.

Thanks in advance.

Potential errors in code

@crashmoon Thanks for your great work! I got some issues about your code: in your file gan_lstm.py , line 338-356 are listed as follows:

class Net_G(nn.Module):
    def __init__(self):
        super(Net_G, self).__init__()
        self.unet_1 = Unet(3, Param.unet_channel * 8)
        self.unet_2 = Unet(6, Param.unet_channel * 8 * 3)
        self.unet_3 = Unet(6, Param.unet_channel * 8 * 3)
        self.unet_4 = Unet(6, Param.unet_channel * 8 * 3)
        self.rnn = nn.LSTMCell(Param.unet_channel * 8, Param.unet_channel * 8 * 2)

    def forward(self, data_1, data_2, data_3, data_4, h0, c0):
        #print(data_1.size())
        unet_out_1, unet_mid_1 = self.unet_1(data_1)
        h1, c1 = self.rnn(unet_mid_1.view(Param.batch_size, -1), (h0, c0))
        unet_out_2, unet_mid_2 = self.unet_2(torch.cat((data_1, unet_out_1), 1), h1)        # check
        h2, c2 = self.rnn(unet_mid_2.view(Param.batch_size, -1), (h1, c1))
        unet_out_3, unet_mid_3 = self.unet_3(torch.cat((data_1, unet_out_2), 1), h2)        # check
        h3, c3 = self.rnn(unet_mid_3.view(Param.batch_size, -1), (h2, c2))
        unet_out_4, unet_mid_4 = self.unet_4(torch.cat((data_1, unet_out_3), 1), h3)        # check
        return unet_out_1, unet_out_2, unet_out_3, unet_out_4

In my opinion, the arguments passed into self.unet_2, self.unet_3, self.unet_4 (I've appended with '# check') should be data_2, data_3, data_4 respectively. Could you please help me check out with this, and correct me if I am wrong, thanks!

train problem

hi ~~first, I creat a file named gan_lstm.csv and i run make-image.py,but there is nothing in it . Then i run gan_lstm.py, a problem happened:
FileNotFoundError: [Errno 2] File b'paris.csv' does not exist: b'paris.csv'
so what should i do ,i just changed Param: out_path. by the way my make_image is
os.chdir('./dataset/paris_train_original/')
name = os.listdir(os.getcwd())
print(name)

id = 0
with open('./gan_lstm.csv','w') as csvfile:
w = csv.writer(csvfile)
for x in name:
w.writerow(['.dataset/paris_train_original/'+x, id])
id+=1

训练mask和测试mask必须保持一致吗？

@crashmoon 感谢您优秀的工作！从您的论文中可以看到，您用了4个生成器逐步补全缺失mask。有一点好奇的是，如果训练的mask和测试的mask不一致，我该融合这四个mask的结果。好比，我用6464的mask训练，测试时我用6448的mask测试，那么第一个生成器得到结果，哪些部分是可以保留的，作为第二阶段的输入？

problem about train

when i run gan_lstm.py,a problem happened.
AttributeError: 'generator' object has no attribute 'next'

can you provide "gan_lstm.csv"?

Could you provide pretrained weights?

Hi,
I would like to test your code in different examples.
Could you provide pretrained weights?

Thanks in advance.