firstmover/pytorch_hmr

pytorch version of "End-to-end Recovery of Human Shape and Pose"

HMR

This is a pytorch implementation of End-to-end Recovery of Human Shape and Pose by Angjoo Kanazawa, Michael J. Black, David W. Jacobs, and Jitendra Malik, accompanying by some famous human pose estimation networks and datasets. HMR is an end-to end framework for reconstructing a full 3D mesh of a human body from a single RGB image. In contrast to most current methods that compute 2D or 3D joint locations, HMR produce a richer and more useful mesh representation that is parameterized by shape and 3D joint angles. The main objective is to minimize the reprojection loss of keypoints, which allow model to be trained using in-the-wild images that only have ground truth 2D annotations. For visual impact, please visit the author's original video.

training step

1. download the following datasets.

• AI challenger keypoint dataset
• lsp 14-keypoint dataset
• lsp 14-keypoint extension dataset
• COCO-2017-keypoint dataset
• mpi_inf_3dhp 3d keypoint dataset
• mosh dataset, which used for adv training

2. download human3.6 datasets.

• hum3.6m_part_1.zip
• hum3.6m_part_2.zip
• hum3.6m_part_3.zip
• hum3.6m_part_4.zip
• hum3.6m_part_5.zip
• hum3.6m_anno.zip

3. unzip the downloaded datasets.

4. unzip the model.zip

5. config the environment by modify the src/config.py and do_train.sh

6. run ./do_train.sh directly

environment configurations.

• install pytorch0.4
• install torchvision
• install numpy
• install scipy
• install h5py
• install opencv-python

result

reference papers

• Stacked Hourglass Networks for Human Pose Estimation
• SMPL: A Skinned Multi-Person Linear Model
• Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
• motion and shape capture from sparse markers
• Unite the People: Closing the Loop Between 3D and 2D Human Representations
• End-to-end Recovery of Human Shape and Pose

reference resources

• up-3d dataset
• coco-2017 dataset
• human 3.6m datas
• ai challenger dataset