wangyujie615/unipose

Official implementation of the paper "UniPose : Detecting Any Keypoints"

UniPose: Detecting Any Keypoints

Project Page | Paper | ReadPaper | UniKPT data(coming soon) |Video

Jie Yang^1,2, Ailing Zeng¹, Ruimao Zhang², Lei Zhang¹

¹International Digital Economy Academy ²The Chinese University of Hong Kong, Shenzhen

🤩 News

• 2023.10.13 : We release the arxiv version.

In-the-wild Test via UniPose

UniPose has strong fine-grained localization and generalization abilities across image styles, categories, and poses.

🗒 TODO

• Release inference code and demo [Expected on October 30th].
• Release checkpoints [Expected on October 30th].
• Release UniKPT dataloader and annotations[Expected on October 30th].
• Release training codes.

💡 Overview

• UniPose is the first end-to-end prompt-based keypoint detection framework.

• It supports multi-modality prompts, including textual and visual prompts to detect arbitrary keypoints (e.g., from articulated, rigid, and soft objects).

Visual Prompts as Inputs

Textual Prompts as Inputs

Citing UniPose

If you find this repository useful for your work, please consider citing it as follows:

@article{yang2023unipose,
  title={UniPose: Detection Any Keypoints},
  author={Yang, Jie and Zeng, Ailing and Zhang, Ruimao and Zhang, Lei},
  journal={arXiv preprint arXiv:2310.08530},
  year={2023}
}

@inproceedings{yang2023neural,
  title={Neural Interactive Keypoint Detection},
  author={Yang, Jie and Zeng, Ailing and Li, Feng and Liu, Shilong and Zhang, Ruimao and Zhang, Lei},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={15122--15132},
  year={2023}
}

@inproceedings{yang2022explicit,
  title={Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation},
  author={Yang, Jie and Zeng, Ailing and Liu, Shilong and Li, Feng and Zhang, Ruimao and Zhang, Lei},
  booktitle={The Eleventh International Conference on Learning Representations},
  year={2022}
}