ingeniousfrog/latent-to-latent

Unofficial PyTorch implementation of "Latent to Latent: A Learned Mapper for Identity Preserving Editing of Multiple Face Attributes in StyleGAN-generated Images"

Latent-To-Latent

Unofficial PyTorch implementation of "Latent to Latent“

---

Latent-To-Latent: A Learned Mapper for Identity Preserving Editing of Multiple Face Attributes in StyleGAN-generated Images

Abstract: Several recent papers introduced techniques to adjust the attributes of human faces generated by unconditional GANs such as StyleGAN. Despite efforts to disentangle the attributes, a request to change one attribute often triggers unwanted changes to other attributes as well. More importantly, in some cases, a human observer would not recognize the edited face to belong to the same person. We propose an approach where a neural network takes as input the latent encoding of a face and the desired attribute changes and outputs the latent space encoding of the edited image. The network is trained offline using unsupervised data, with training labels generated by an off-the-shelf attribute classifier. The desired attribute changes and conservation laws, such as identity maintenance, are encoded in the training loss. The number of attributes the mapper can simultaneously modify is only limited by the attributes available to the classifier – we trained a network that handles 35 attributes, more than any previous approach. As no optimization is performed at deployment time, the computation time is negligible, allowing real-time attribute editing. Qualitative and quantitative comparisons with the current state-of-the-art show our method is better at conserving the identity of the face and restricting changes to the requested attributes.

---

Pre-trained Networks

Path	description
Restyle_e4e_ffhq_encode	ReStyle applied over e4e trained on the FFHQ dataset.
[BranchedTiny](链接：https://pan.baidu.com/s/1aOvZjYG6xepgCr0B6cnvJA?pwd=hwv6 提取码：hwv6 --来自百度网盘超级会员V1的分享)	pretrained model for Attribute regressor
[mobile_facenet](链接：https://pan.baidu.com/s/1edI6edYuDaeiTKWzDs4JnQ?pwd=r6dn 提取码：r6dn --来自百度网盘超级会员V1的分享)	pretrained model for face_extractor

The pre-training files under mobile_facenet need to be placed in the project utils/FaceExtractor/checkpoint directory

BranchedTiny.ckpt needs to be placed in the project models\attribute_classifier directory

---

Training

I have trained smile and bushy_eyebrows attributes on FFHQ dataset. The remaining attributes can be trained by themselves.

python trainer.py \
--exp_dir ./ffhq
--attribute young
--gpu 0
--epoch 10
--batch_size 8

The kind of attribute which you can train depand on the attribute_classifier.

attribute documentation : attribute documentation

Testing

Inference

python inference.py \
--attribute smiling \
--gpu 0 \
--ckpt ./pretrained/smiling.pt \
--exdir ./expir \
--savedir ./result

Other

Since the project is not open source, and the author has too little description of the model in the paper, the implementation of some modules in the model is my own replacement.

（1）Attribute regressor

The author used Microsoft API to generate data attribute scores for a large number of images and trained attribute classifiers. I used an open source attribute classifier as an alternative.

（2） Latent-To-Latent Model

The authors only describe the use of a fully connected network, but do not go into detail. In my experimental scheme, I used a 2-layer fully connected network, and at the same time borrowed from StyleClip's scheme and used an inverted pyramid structure.

If you have a better solution or suggestion, please post an issue message.