csmliu / stgan Goto Github PK
View Code? Open in Web Editor NEWSTGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
License: MIT License
STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
License: MIT License
can you test images outside the test celebA test set?
Hey, Thanks for your wonderful work. I am not getting a proper bald image in the output as given by you in the paper. can you please guide me to improve the accuracy of the bald image in the output
Hi, thank you for sharing the code. I am now trying to train stgan with only 24 attributes. I have the aligned dataset and the corresponding labelset. Which part in your code do I need to change?
Thank you.
hello,
i have downloaded trained model and trying to test it on google colab but i am getting index error in data file
i have used only 637 images from 182000 to 182637
img_paths = [img_paths[i-1] for i in im_no]
IndexError: list index out of range
你好,我运行pretrained model处理了img_align_celeba(128)数据集里最后几百张图片,按照你提供的#20 (comment) 评分代码,运行出来结果是29.181917111413046和
0.8883383051577889,当然这已经是非常高的分数了,只是和你论文里提到的稍有一丝差距,请问该怎么解决;
另外叨扰下,如果要测你的模型的FID和IS分数该如何准备测试用的结果数据集
Hello,
I am trying to get the pretrained model working and running into some difficulties. In the repository you suggest that we should refer to the Attgan-tf repository for dataset preparation. However that repository seems to be using the non-alligned dataset and cropping it while you suggest to use the aligned. I was wondering if you could clarify what steps from that repository need to be taken in order to get the pretrained model working
Thanks for sharing the model!
I reconstructed the data.py with my dataset(80K training images), and used this framework, everything went well except the time cost. The model spent nearly one day training only an epoch with two gpus. I wonder how many gpus you have used in your experiments and how much time have you spent.
Thanks a lot!
Use tf.data.Dataset.batch(..., drop_remainder=True)
.
NOTE: Initializing all parameters...
Traceback (most recent call last):
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [height must be >= target + offset.]
[[{{node crop_to_bounding_box/Assert_1/Assert}} = Assert[T=[DT_STRING], summarize=3](crop_to_bounding_box/GreaterEqual_1, crop_to_bounding_box/Assert_1/Assert/data_0)]]
[[{{node IteratorGetNext}} = IteratorGetNextoutput_shapes=[[32,128,128,3], [32,13]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
[[{{node RandomShuffle/_15}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_156_RandomShuffle", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 337, in
d_summary_opt, _ = sess.run([d_summary, d_step], feed_dict={lr: lr_ipt})
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/lw/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [height must be >= target + offset.]
[[{{node crop_to_bounding_box/Assert_1/Assert}} = Assert[T=[DT_STRING], summarize=3](crop_to_bounding_box/GreaterEqual_1, crop_to_bounding_box/Assert_1/Assert/data_0)]]
[[node IteratorGetNext (defined at /home/lw/workspace/STGAN/data.py:102) = IteratorGetNextoutput_shapes=[[32,128,128,3], [32,13]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
[[{{node RandomShuffle/15}} = Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_156_RandomShuffle", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
Model is saved at ./output/128/checkpoints/Epoch(0)(1of947).ckpt!
Why do you use val_data When you train ????
Hello, is there a tensorflow version of the code for the quantitative analysis part of the paper?
Hello, I really want to know where the PSNR/SSIM code is in the quantitative analysis in the paper?
请问我在运行show_images.py时遇到KeyError: 'Ours'该如何解决
Hi, csmliu!
Thanks for your excellent work which helps me l lot!
But when i try to reproduce 384x384 result , I always can not to get such good effect in your readme.md file. for example, to old and to remove bangs. Could you tell me some train tricks of 384x384?,I follow the LynnHo/HD-CelebA-Cropper to crop 384 image from celeba origin image. and the train scipt is:
python ../../train.py
--dataroot
--list_att_file new_list_attr_celeba_addhair.txt
--batch_size 16
--gpu 0
--experiment_name 384
--img_size 384
--enc_dim 48
--dec_dim 48
--dis_dim 48
--dis_fc_dim 512
--n_sample 24
--use_cropped_img \
Could you help me for how to reproduce result in ReadMe.md? and i test my model of epoc35, and after epoc40, model didnt change better. Waiting for your apply~
Thanks for your excellent work which helps me l lot!
Recently,I want to use mode of lsgan or dcgan to train,but the code has something wrong.Could you please help me how to solve it?
Thank you very much.
Traceback (most recent call last):
File "train.py", line 205, in
gp = models.gradient_penalty(D, xa)
File "/home/dell/STGAN-master/models.py", line 157, in gradient_penalty
x = _interpolate(real, fake)
File "/home/dell/STGAN-master/models.py", line 148, in _interpolate
_, variance = tf.nn.moments(a, range(a.shape.ndims))
File "/home/dell/anaconda3/envs/test/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 1028, in moments
mean = math_ops.reduce_mean(y, axes, keepdims=True, name="mean")
File "/home/dell/anaconda3/envs/test/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/home/dell/anaconda3/envs/test/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1764, in reduce_mean
name=name))
File "/home/dell/anaconda3/envs/test/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 6180, in mean
name=name)
File "/home/dell/anaconda3/envs/test/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 545, in _apply_op_helper
(input_name, err))
ValueError: Tried to convert 'reduction_indices' to a tensor and failed. Error: Argument must be a dense tensor: range(0, 4) - got shape [4], but wanted [].
Hi Ming, I am trying to run training on a 8 GPU cluster and have changed the --gpu settings appropriately. However I see that the training is still only using 1 GPU, namely "0". I can dig deeper into how the device is being distributed into training, but am wondering if I just did something easily fixable. Thanks!
Hi, which file do you use to compute the psnr/ssim of your results?
Thank you.
Hi, here is a question about the the attribution classification results of AttGAN in your paper.
Are the results tested on the released model of AttGAN-Tensorflow ? Or you just retrain a new AttGAN model and test on it.
I test the 128x128 model released by AttGAN-Tensorflow, and get different results from yours. As for your results, I refer to this file quantitative.results.xlsx.
If I want to change the dataset and train it, how is the list_attr_celeba.txt file created?
hi,could u provide me the code and data about season translation,please。Nowadays,we have improved your work (STgan), and could we have a talk about the detail of this work? my wechat account is gxw18061037210.please,get in touch with me.thank u so much!
在Github上提供的预训练模型是专门在CelebA数据集上进行14个属性转换的,如果我需要在其他的数据集上进行属性转换,我需要如何修改代码?这个预训练模型可以为其他数据集的属性转换重新训练吗?
please tell me How to evaluate PSNR/SSIM?
Hello,csmliu!
First of all ,thank you so much for your brilliant work which is inspiring! : )
But I got some problem when I try to calculate the Attribute Generation Accuracy .
In your paper,the average Attribute Generation Accuracy is about 90%. A few days ago,I generated the sample_testing image by using the pre_trained model you provided,and measured the Attribute Generation Accuracy by using discriminator of the pre_trained model.
However , the accuracy i got is far less than the accuracy you proposed in the paper.
Hi , When I use thr att classifcation , it broke at "imgs = np.concatenate(np.split(imgs, 15, axis=2))" And I also very puzzle at
imgs = np.concatenate([imgs[:, :img_size, :], imgs[:, img_size+img_size//10:, :]], axis=1)
imgs = np.expand_dims(imgs, axis=0)
imgs = np.concatenate(np.split(imgs, 15, axis=2))
Why should do this?
Can I get the attributes like the face direction?
According to the paper, you used an environment of 'Intel(R) Xeon(R) E3-1230v5 CPU 3.40GHz and Nvidia GTX1080Ti GPU' to train STGAN and your celeba dataset has 200K images.
May I ask how long it takes you to train the whole network (I remember that your pretrained network was trained for 139 epochs)? Thank you.
hello, if I use the 512*512 to training , does it working ok ?
hello,Could you please tell me the detailed data of IcGAN. Fadernet ,attgan and your STGAN? You only show a table without data, THANKS!
I am curious about the effect if you replace BN to IN for generator, as is common practice in recent Style Transfer researches. Have you tried this?
Thanks for the impressive work!
Hi, csmliu, thank you so much for your brilliant work!
I want to use STGAN as a baseline in my paper and want to get the attribute classification accuracy of STGAN on some attributes instead of your predefined attributes. But I find it confusing about the required tfrecord data, which is a little different from LynnHo/TfrecordCreator. Just want to avoid potential error, could you give a more detailed tutorial for the training of attribute classifier?
scipy.misc.imsave has been deprecated in newer Scipy versions and on Ubuntu 18.04 (Scyipy version 1.3.3) it gives attribute error. You can change it by any other saving function, I used imageio.imwrite function with the same parameters (from imageio library).
Thank you for your excellent work at first
I cloned this repo and followed the readme, everything works fine, and i would like to know if there's anyway to change only one attribute of a face(young to old for example) and generatr only one image?
Thank you for your kind help in advance!
怎么指定自己的测试图片,还有就是结果里面变老的属性是什么?
测试的配置文件里面只有这几个属性:
"atts":[
"Bald",
"Bangs",
"Black_Hair",
"Blond_Hair",
"Brown_Hair",
"Bushy_Eyebrows",
"Eyeglasses",
"Male",
"Mouth_Slightly_Open",
"Mustache",
"No_Beard",
"Pale_Skin",
"Young"
],
还有一个问题是测试的时候为什么还需要调用list_attr_celeba.txt
文件,这个不应该是训练才要用到的标签吗?
希望得到您的帮助,非常感谢!
Hello @csmliu
I have a question.I have converted the model into frozen_graph.pb how to check the generated model has the same accuracy as the original model?
Do u have any experience of tflite?
If yes, then I want to know,if I want to deploy the model on Android then can I use the .pb file directly or do I need to convert it into tflite?
Thank you for this work. I have a question about Gpu. how to train this code using Gpu i have Rtx 2070 and try to install tensorflow-gpu but there are error and when use tensorflow 2.0 also error. How to solve this problem because i need to use my gpu
The images in img_align_celeba are of 178218. I tried to train the network for testing 128128 images. I resized and aligned the images in the dataset. I trained the network using 'python train.py --experiment_name 128'.
However, I keep getting errors as below.
Traceback (most recent call last):
File ".local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
return fn(*args)
File ".local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File ".local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [height must be >= target + offset.]
[[Node: crop_to_bounding_box/Assert_1/Assert = Assert[T=[DT_STRING], summarize=3](crop_to_bounding_box/GreaterEqual_1, crop_to_bounding_box/Assert_1/Assert/data_0)]]
[[Node: IteratorGetNext_1 = IteratorGetNextoutput_shapes=[[64,128,128,3], [64,13]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "STGAN/train.py", line 312, in
xa_sample_ipt, a_sample_ipt = val_data.get_next()
File "STGAN/data.py", line 93, in get_next
return self._sess.run(self._batch_op)
File "/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File ".local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in do_run
run_metadata)
File ".local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [height must be >= target + offset.]
[[Node: crop_to_bounding_box/Assert_1/Assert = Assert[T=[DT_STRING], summarize=3](crop_to_bounding_box/GreaterEqual_1, crop_to_bounding_box/Assert_1/Assert/data_0)]]
[[Node: IteratorGetNext_1 = IteratorGetNextoutput_shapes=[[64,128,128,3], [64,13]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Traceback (most recent call last):
File "STGAN/train.py", line 381, in
save_path = saver.save(sess, '%s/Epoch(%d)(%dof%d).ckpt' % (ckpt_dir, epoch, it_in_epoch, it_per_epoch))
NameError: name 'it_in_epoch' is not defined
Process finished with exit code 1
I thought it might be the problem with changing the input image. I went back to your original dataset and still met the same errors.
Could you please give me some suggestions about my problems? Thank you.
Hello!
First of all I want to thank you for this very interesting work and of course for sharing your implementation!
I was trying to reproduce your season translation results (yosemite dataset) but cant get it working.
I made a separate dataset class similar to the existing Celeba one to load the yosemite data, set the att_dict to {'summer': 0, 'winter': 1} and specified label pairs for each image (e.g. [1, -1] summer, [-1, 1] winter).
I tried training the STGAN with diff and target label parameter, but it did not help. The generated images during training (sample_training) as well as when using the test function always show the identical input images.
Is there a special parameter configuration needed to achieve image2image translation results?
Would you mind sharing your implementation for this experiment too?
Thank you in advance!
Hi, csmliu
At present, the results of training on high-resolution images are not good. Is there any other way to improve the conversion results of high-resolution images, such as 512 or 1024 images?If I add a data set of high resolution images, will it improve the current effect at high resolution?
您好~
我在ATTGAN裡面有看到可以選擇眼鏡的樣式,
而STGAN是由ATTGAN更改的,
請問這個程式可以透過任何方法自由選取眼鏡樣式嗎?
希望能得到您的幫助,非常感謝!
Hello,
First of all, Thank you for providing your code publicly.
Secondly, How can i use the pre-trained model to see the output of the network.
Many thanks.
hi, thanks for your great project.
I have a question about the inference of my images.
how to change the attributes of my images by using the model?
excuse me, why u think that the epochf139 is the best one ? and how to search it ? try everything possible?! expect your reply, thank u!
Hello, where can I find the code for PSNR/SSIM in quantitative analysis?
Hi, thank you for sharing your project.
Do you think it is possible to add noise to interfere with the generation of desired attribute like glasses in order to have varied attribute(glasses)'s style?
Along with the previous question, I visualized the nodes in different parts of the model. It seems that glasses are generated in STU, so I think it might work if I add noise just one step before the glasses (information or shape of it) are generated. Could you give me some idea where should be the place the attribute is first generated (there are dozens of nodes in STU and there are four STUs)?
Thank you!
I want to removal the eyegalsses on my own face image. I find use the model which training on CeleA will not work. So, I use own data to train the model. However, it still not work. The image looks like the reconstruction image which the attribute about eyeglasses is not be changed. I want to get some suggestion about training the model. How many images should be prepared for training. I only use 40000 images with eyeglasses and without eyegalsses
您好!
我做了添加鬍子的測試,請問有任何方法選取鬍子的樣式或是調整它的濃密強度嗎?
感謝您的幫助!
你好,想请教下,在训练之前将训练数据的标签的某一位所代表的属性就行指定的reverse,比如只改变训练数据的眼镜这一位对应的标签,然后他们之间的差异就是眼镜这个属性放进去训练,然后这个模型就指定转换眼镜的差异。我修改的代码如下,但是最终训练效果好像不是太好,不知道是什么原因。在使用验证集数据的时候作了同样的修改,但是最终验证没有效果。
代码:
xa, a = tr_data.get_next()
b=copy.deepcopy(a)
for i in enumerate(b.shape[0]):
b[i][6] = 1-b[i][6]
# b = tf.random_shuffle(a)#将标签随机打乱
xa = tf.convert_to_tensor(xa)
a = tf.convert_to_tensor(a)
b = tf.convert_to_tensor(b)
_a = (tf.to_float(a) * 2 - 1) * thres_int
_b = (tf.to_float(b) * 2 - 1) * thres_int
请问n_d=5是最优的策略吗?
Hi, Thank you for the good work! I read your interesting work.
After reading the paper and the code, I have a question about the attribute vector.
My understanding is that your code uses a different value of attribute vector in the training and test phase. (I read all of the issues including #13 #18 #21 etc.)
The explanation about my understanding :
In the training phase, you might use the attribute vector with range [-0.5, 0.5] (https://github.com/csmliu/STGAN/blob/master/train.py#L177, read [0, 1] then (*2 - 1) * thres_int => [-1, 1] * 0.5 = [-0.5, 0.5] ) Therefore, the difference vector ranges [-1, 1] (-1 or 0 or 1)
In contrast, in the test phase, you seem to use the attribute vector ranged [-1.5, 1.5]. (https://github.com/csmliu/STGAN/blob/master/train.py#L360, read [0, 1] then (*2 -1) * thres_int * test_int / thres_int => [-1, 1] for _b_sample_ipt and [-0.5, 0.5] for raw_b_sample)
Therefore, test_label in L280 ranges [-1.5, 1.5]
For example, in training, a_ = [-0.5, 0.5, -0.5] , b_ = [0.5, -0.5, -0.5] then, the difference vector is
b_-a_ = [1.0, -1.0, 0.0].
In the test phase, test_label = _b_sample - raw_b_sample
Let b_sample_ipt_list[0] = [1, 0, 1], then raw_b_sample = [0.5, -0.5, 0.5] and b_sample_ipt_list[1] = [0, 0, 1].
Therefore, _b_sample_ipt[1] = [-0.5, -0.5, 0.5] in L358.
In L360, _b_sample_ipt[1] = [-1.0, -0.5, 0.5].
So, test_label[1] = [-1.0, -0.5, 0.5] - [0.5, -0.5, 0.5] = [-1.5, 0.0, 0.0]
Actually, I cannot reproduce the results without the trick. However, I found that the trick drastically changes the performance of the model.
Is my understanding correct? If so, please let me know. (I just want to use your work as a baseline for my future research. So I must clearly understand your work.)
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.