kaito0311/dsfnet

Code for DSFNet: Dual Space Fusion Network for Occlusion-Robust Dense 3D Face Alignment

@kaito0311 note

Add code generate BFM data by python code and visualize result.

DSFNet

Paper link: https://arxiv.org/abs/2305.11522
Project link: https://lhyfst.github.io/dsfnet/

Requirements

python                    3.6.13
pytorch                   1.7.1
cudatoolkit               10.1.243
imageio                   2.15.0
numpy                     1.19.2
opencv-python             4.7.0.72
PyYAML                    6.0
scikit-image              0.17.2
torchvision               0.8.2
tqdm                      4.64.1
trimesh                   3.22.1

You can easily prepare the conda environment by conda create --name DSFNet --file requirements.txt

Prepare

• Please refer to face3d to prepare BFM data. And move the generated files in Out/ to data/Out/ . Can use convert.py to use instead file generate matlab

• Download BFM_UVspace_patch.npy. Put it under data/uv_data/

• Download pretrained model. Put it under data/saved_model/.

Testing for only image

• Structure folder : data/AFLW2000_crop/image0001/- {image0001.jpg} and {image0001_pos_map.npy} (can fake or remove in code because it used only for evaluation)

Evaluation

• Download AFLW2000-3D at http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3ddfa/main.htm .

• Follow SADRNet to crop images and prepare the image directory. Or you can download the cropped images at link. Put them at data/dataset/AFLW2000_crop.

• Run src/run/predict.py. In the returned text, nme3d, rec, MAE are the results of dense 3D dense face alignment, reconstruction, and head pose estimation.

Acknowledgements

We especially thank the contributors of the SADRNet codebase for providing helpful code.