take2rohit/bedlam

BEDLAM: Bodies Exhibiting Detailed Lifelike Animated Motion

CVPR 2023

Project Page | Paper | Video | Render code

^{Recounstruction results on images from different benchmarks: HBW, SSP-3D, RICH.}

This repository contains the code to train and evaluate BEDLAM-CLIFF, BEDLAM-HMR, BEDLAM-CLIFF-X model from the paper. If you are interested in the Unreal code for rendering synthetic images, please check out the repository here. To process the data generated by Unreal into a format that could be used for training please checkout data_processing section.

News

2023/07/04: Converted SMPL ground truth labels for training available on project page.

Install

Create a virtual environment and install all the requirements

python3.8 -m venv bedlam_venv
source bedlam_venv/bin/activate
pip install -r requirements.txt

Quick Demo

Prepare data

If you need to run just the demo, please follow the following steps:

Step 1. Register on SMPL-X website.

Step 2. Register on MANO website.

Step 3. Register on BEDLAM website.

Step 4. Run the following script to fetch demo data. The script will need the username and password created in above steps.

bash fetch_demo_data.sh

BEDLAM-CLIFF demo

 python demo.py --cfg configs/demo_bedlam_cliff.yaml

BEDLAM-CLIFF-X demo

python demox.py --cfg configs/demo_bedlam_cliff_x.yaml --display

Dataset visualization

Once you download BEDLAM dataset following the instructions in training.md, you can use the script to visualize the projection of 3D bodies on images

python visualize_ground_truth.py output_dir

Evaluation

For instructions on how to run evaluation on different benchmarks please refer to evaluation.md

Training

For instructions on how to run training please refer to training.md

BEDLAM leaderboard

If you want to upload your results to BEDLAM evaluation server, please follow the instructions here.

Citation

@inproceedings{Black_CVPR_2023,
  title = {{BEDLAM}: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion},
  author = {Black, Michael J. and Patel, Priyanka and Tesch, Joachim and Yang, Jinlong}, 
  booktitle = {Proceedings IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
  pages = {8726-8737},
  month = jun,
  year = {2023},
  month_numeric = {6}
}

License

Please checkout the license here. Questions related to licensing could be addressed to [email protected]

References

We benefit from many great resources including but not limited to SMPL-X, SMPL, PARE, CLIFF, AGORA, PIXIE, HRNet.