charliememory / pose-guided-person-image-generation Goto Github PK
View Code? Open in Web Editor NEWTensorflow implementation of our NIPS 2017 paper "Pose Guided Person Image Generation"
License: MIT License
Tensorflow implementation of our NIPS 2017 paper "Pose Guided Person Image Generation"
License: MIT License
Download links on your "Resources" are not work!
Please fix it~~
when I run the commond of "./run_convert_market.sh" in TF-record data preparation steps and the result is as follows:
100000
200000
repeat positive pairs augment_ratio times and cut down negative pairs to balance data ......
p_pairs length:296938
n_pairs length:41936
save p_pairs and n_pairs ......
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 911, in run_one_pair_rec
mode='same_diff_cam')
File "convert_market.py", line 202, in _get_train_all_pn_pairs
pickle.dump(p_pairs,f)
TypeError: write() argument must be str, not bytes
OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 32, current size 0) [[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]] [[Node: batch/_1515 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_1 0_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
I select several images from test clean set (not in test sample set), and convert the images by your scripts. But I met the above bug during the inference. Could you give me any suggestions?
Hi~I get a 403 Forbiddien when I want to download the training data in tf-reocrd format.And it looks like all the resources can't be downloaded.Can you please fix it?
According to the article, The pose estimator generates the coordinates of 18 keypoints, then you encode pose as 18 heatmaps
but I can't find the function which converts the coordinates to heatmaps.
can you help me, please?
According to your code, generating 18 keypoints depends on file——annotation - {type}. csv,
so how to generate my own data set's xxxx.csv file?
Hi. When i tried to process data with "run_convert_DF.sh" i get this error. Do you know how i can solve it?
Thanks
File "convert_DF.py", line 1005, in <module>
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_DF.py", line 982, in run_one_pair_rec
mode='same_diff_cam')
File "convert_DF.py", line 172, in _get_train_all_pn_pairs
pickle.dump(p_pairs,f)
TypeError: write() argument must be str, not bytes
Hi,
I could not access any of the datasets / trained models. The server is returning ERROR 403: Forbidden. Will you please fix the problem again? Thanks!
I'm aware this has happened before and was addressed. The issue is back again.
You said you used a pose estimator to generate the target pose and transformed them into 18 heatmaps. Is that procedure included in this code? Or the dataset you offered already contains the target pose?
In the market1501 dataset, I am confused about the meaning of bounding_box_test_clean.
Thanks for your time.
Hi,
Thank you for your code. :)
May I know the intuition behind no batch normalization in your work?
If I want to use other image to test the model performance, how would I do?
Thank you very much
Hi, I am porting PG2 to pytorch.
I am confused about the design of Discriminator:
In your code, the last layer of Discriminator is a linear whose output maybe not in [0, 1].
so, I add a sigmoid to output:
self.fc = nn.Linear(8*4*8*base_channels, 1)
self.sigmoid = nn.Sigmoid()
self.main.apply(weights_init)
def forward(self, x):
x = self.main(x)
x = x.view(-1, 8*4*8*self.base_channels)
return self.sigmoid(self.fc(x))
copy from here
if I only change the input layer's stride to (4, 2) of DCGAN's Discriminator.
nn.Conv2d(in_channels=nc, out_channels=nf, kernel_size=4, stride=(4, 2), padding=1, bias=False),
nn.LeakyReLU(0.2, inplace=True),
I get a better output.
but it's still far from your result in paper.
@charliememory Thanks for your patience. ( ̄▽ ̄)", I have implemented your paper with Pytorch.
Pytorch implemention: Human-Pose-Transfer
result:
https://github.com/budui/Human-Pose-Transfer#implement-result
In addition, I also implemented PatchGAN-PG2
, LSLoss-PG2
@charliememory Once I have trained the model, I want to run Pose Transfer on a given image. But the current implementation required a tfrecord file to be generated from images and poses, and applies pose transfer on those. Any quick solution for that ?
Since your datasets format is .p, I do not know how to make the training datasets with my image, Can you talk about the detail (like parameters)of the datasets format?
Thank you very much.
What is the intuition behind using the triplet scheme as compared to the pair wise scheme used in the paper? What kind of improvements does it lead to?
When i tried to run testing with your provide pretrained model. I get this error
Traceback (most recent call last):
File "score_mask.py", line 167, in <module>
masked_G1_array = np.uint8(mask_target_list[i][:,:,np.newaxis]/255.*G1_list[i])
IndexError: list index out of range
Do you know how to solve this problem?
Thanks
Hi, I'm trying to download the resources but I got :
You don't have permission to access /~liqianma/NIPS17_PG2/models/DF_model.zip on this server.
Could you please fix the problem?
Thanks!
I can't find the supplementary material mentioned in the paper,could you give me give a link, thank you!
Even after lots of debugging, the downloaded pre-trained model cannot be loaded with the following error:
DataLossError (see above for traceback): Unable to open table file ./pretrain_models/PG2_model_D│face_morpher Sketch-Photo-Conversion-using-Deep-CNN
F/model.ckpt-0.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your f│FastPhotoStyle SketchToFace
ile is in a different file format and you need to use a different restore operator?
There is an error when I run run_DF_test.sh.
AttributeError: 'PG2_256' object has no attribute 'G'
I think we have to assign self.G as self.G2 in the build_model model of PG2_256 in trainer256.py:
self.G1 = denorm_img(G1, self.data_format)
self.G2 = denorm_img(G2, self.data_format)
self.G = self.G2 # Add this to fix no attribute 'G' error
@charliememory I am trying to freeze the model from pbtxt and ckpt files but can not able to find output and input node name. Can you please tell how to freeze the model graph and use it to convert to tensorflow lite model. Also if you have frozen graph can you please share ?
I have trained the model and want to use that for testing which ckpt file am i supposed to use for testing as i think the run_DF_test file was meant for pretrained model
I would like to know what are the required system resources (i.e. CPU, GPU, RAM), in order to train and test this model?
Hi, Liqian,
Thanks for your open-source code of your project. I have few questions about some details in your network:
Q1: In [1]:
I note that in the decoders of G1 and G2, you use upscale + conv
to up-sample the feature map, and you comment a line in which transposed conv
is used to up-sample. These two methods, do they have big influence on the final performance?
Just see code here
Q2:Based on Q1, which of method you used in your paper [2]?
Q3: In [2]:
In Sec 3.4, what the expressions "fully-connected residual layers"
mean?
Because your illustrations in supplementary materials looks that the "mapping functions"
and "Pose auto-encoder"
just consists of simple fully-connections. I haven't found any "residual"
elements.
Looking forward to your reply.
References:
[1]Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., & Van Gool, L.. Pose Guided Person Image Generation. NIPS2017
[2]Disentangled Person Image Generation
Dataset and tf_records links are broken
I am trying to train the model with another dataset Lookbook. The inputs are image_raw, target_image, pose and mask, the size of target pose is (256,256,9). I got the following error when start training. It happened at this line in function get_image_from_loader(self) (i modified your code):
x, x_target, pose_target, mask_target = self.sess.run([self.x, self.x_target,self.pose_target, self.mask_target])
OutOfRangeError (see above for traceback): FIFOQueue '_4_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /home/ychen/git/Pose-Guided-Person-Image-Generation/train_lookbook.py:114) = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]
I am quite new to Tensorflow, any help is much appreciated!
Hi! Thanks for your open-source code of your project. I have few questions about some details in trainer.py.
Why do you cycle 400 times in the test procedure?
Is every iteration better than the previous one?
Thanks a lot!
How to use pretrained model for single image pose change
I have made changes in the lines:
By replacing o._shape = tf.TensorShape(new_shape) with o.set_shape (tf.TensorShape(new_shape))
and logits = tf.matmul(tf.squeeze(pool3), w) with logits = tf.matmul(tf.squeeze(pool3,[1,2]), w) to remove the previous errors but Now I am facing the below value error:
Traceback (most recent call last):
File "score_mask.py", line 107, in
IS_G_mean, IS_G_std = tflib.inception_score.get_inception_score(G_list_masked)
File "/mocapdata/Students/sheela/Pose-Guided-Person-Image-Generation/tflib/inception_score.py", line 46, in get_inception_score
pred = sess.run(softmax, {'ExpandDims:0': inp})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 930, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1129, in _run
str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (100, 256, 256, 3) for Tensor u'ExpandDims:0', which has shape '(1, ?, ?, 3)'
It would be really helpful if someone can give me the solution?
"You don't have permission to access ..."
Would you please to fix it?
Would you explain the steps involved in dataset preparation including the data pairing for training dataset. I want to train my own data. So it would be helpful. Looking for yours positive response.
Hi,
Thanks for your quickly update code.
I found that there might exist some errors in run_DF_test.sh
and run_DF_train.sh
.
if [ ! -d ./data/DF_train_data ]; then
cd data
wget homes.esat.kuleuven.be/~liqianma/NIPS17_PG2/data/DF_train_data.zip
unzip DF_train_data.zip
rm -f DF_train_data.zip
cd -
fi
run_DF_test.sh
and run_DF_train.sh
check /data/DF_train_data
and /data/DF_test_data
respectively. If the folders they wanted do not exist, they will download DF_train_data.zip
and DF_test_data.zip
. However, I checked the DF_train_data.zip
and DF_test_data.zip
which I downloaded several hours ago - I do not sure if you updated those files recently. I found that the the folder structures in DF_train_data.zip
and DF_test.zip
are totally different to the run_DF_train.sh
and run_DF_test.sh
Archive: DF_train_data.zip
Length Date Time Name
--------- ---------- ----- ----
0 2017-09-29 15:58 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/
2657209 2017-09-29 02:20 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/n_pairs_train.p
123995280280 2017-09-29 11:37 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/DeepFashion_train_00000-of-00001.tfrecord
9 2017-09-29 02:20 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/pn_pairs_num_train.p
3064685 2017-09-29 02:20 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/p_pairs_train.p
9 2017-09-29 11:37 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/tf_record_pair_num.txt
--------- -------
124001002192 6 files
Archive: DF_test_data.zip
Length Date Time Name
--------- ---------- ----- ----
0 2017-11-09 21:44 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/
21644244963 2017-11-09 21:37 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/DeepFashion_test_00000-of-00001.tfrecord
726567 2017-11-09 19:48 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/p_pairs_test.p
339187 2017-11-09 19:48 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/n_pairs_test.p
8 2017-11-09 19:48 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/pn_pairs_num_test.p
9 2017-11-09 21:37 data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/tf_record_pair_num.txt
--------- -------
21645310734 6 files
Why has mask_target not been used for stage 1? The paper mentions it's use there. I tried to train a model without mask_target in stage 1 and the results have not been as good as the ones shown in the paper.
Hi,
I've used these settings that you specify in the run_DF_train.sh and could not reproduce the results as provided in your pretrained model link DeepFashion. The results from testing your pretrained model(model.ckpt-0) look much better than what I got from training(model.ckpt-77999) and using it for testing.
gpu=0
D_arch='DCGAN'
stage=1
model_dir="./models/train"
## Make sure dataset name appear in --dataset (i.e. 'Market' or 'DF')
python main.py --dataset=DF_train_data \
--img_H=256 --img_W=256 \
--batch_size=1 --max_step=80000 \
--d_lr=0.00002 --g_lr=0.00002 \
--lr_update_step=50000 \
--is_train=True \
--model=11 \
--D_arch=${D_arch} \
--gpu=${gpu} \
--z_num=64 \
--model_dir=${model_dir} \
When I load the pre-trained DeepFusion model, it encounters shape mismatch.
Pretrained model: UAEnoFC/G/conv18: [3, 3, 256, 128]
current sesson graph: UAEnoFC/G/conv18: [3, 3, 128, 128]
Could you please clarify the graph definition in trainer.py?
Hi!
I can't download your dataset because of forbidden error. Can you please restore an access to it?
Thanks in advance.
What is the no. of epochs and hidden layer in this project?
Hi~When I want to run the run_DF_train.sh
,I got a OutOfRangeError:
OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, b batch/n)]]
,Some TraceBacks:
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]
Caused by op u'batch', defined at:
File "main.py", line 37, in <module>
main(config)
File "main.py", line 24, in main
trainer = PG2_256(config)
File "/Pose-Guided-Person-Image-Generation-master/trainer256.py", line 19, in __init__
self.x, self.x_target, self.pose, self.pose_target, self.mask, self.mask_target = self._load_batch_pair_pose(self.dataset_obj)
I do exactly as your training steps
,Can you give me some advice to fix it?
Thanks~
Now I want to test the code with other datasets,
But I don't know how Market1501_test_00000-of-00001.tfrecord, n_pairs_test.p, p_pairs_test.p, pn_pairs_num_test.p are produced in the Market_test_data.zip folder.
I want to download the resource such as datasets,etc to complete my experiments. But there is a 403 forbidden error. Can you fix it? it might be helpful. Thanks!
Hi,
I am unable to identify the function for pose estimation. I believe that the tf-records of test data already contain precomputed pose information (Correct me if that is not the case). Could you please direct me to the pose estimation and mask estimation function?
Thanks
the datasets folder is referred as data in the run_DF_test.sh script.
in the config.py, the default datasets folder is set to 'data'.
When running the script, it complains that the 'data' folder can not be found.
When I use the downloaded test data set and test with the trained model, the generated image is not the same as the target pose.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.