hualuluu/mmaction

An open-source toolbox for action understanding based on PyTorch

MMAction

Introduction

MMAction is an open source toolbox for action understanding based on PyTorch. It is a part of the open-mmlab project developed by Multimedia Laboratory, CUHK.

Major Features

• MMAction is capable of dealing with all of the tasks below.

  • action recognition from trimmed videos
  • temporal action detection (also known as action localization) in untrimmed videos
  • spatial-temporal action detection in untrimmed videos.

• Support for various datasets

  Video datasets have emerging throughout the recent years and have greatly fostered the devlopment of this field. MMAction provides tools to deal with various datasets.

• Support for multiple action understanding frameworks

  MMAction implements popular frameworks for action understanding:

  • For action recognition, various algorithms are implemented, including TSN, I3D, SlowFast, R(2+1)D, CSN.
  • For temporal action detection, we implement SSN.
  • For spatial temporal atomic action detection, a Fast-RCNN baseline is provided.

• Modular design

  The tasks in human action understanding share some common aspects such as backbones, and long-term and short-term sampling schemes. Also, tasks can benefit from each other. For example, a better backbone for action recognition will bring performance gain for action detection. Modular design enables us to view action understanding in a more integrated perspective.

License

The project is release under the Apache 2.0 license.

Updates

OmniSource Model Release (22/08/2020)

• We release several models of our work OmniSource. These models are jointly trained with Kinetics-400 and OmniSourced web dataset. Those models are of good performance (Top1 Accuracy: 75.7% for 3-segment TSN and 80.4% for SlowOnly on Kinetics-400 val) and the learned representation transfer well to other tasks.

v0.2.0 (15/03/2020)

• We build a diversified modelzoo for action recognition, which include popular algorithms (TSN, I3D, SlowFast, R(2+1)D, CSN). The performance is aligned with or better than the original papers.

v0.1.0 (19/06/2019)

• MMAction is online!

Model zoo

Results and reference models are available in the model zoo.

Installation

Please refer to INSTALL.md for installation.

Update: for Docker installation, Please refer to DOCKER.md for using docker for this project.

Data preparation

Please refer to DATASET.md for a general knowledge of data preparation. Detailed documents for the supported datasets are available in data_tools/.

Get started

Please refer to GETTING_STARTED.md for detailed examples and abstract usage.

Contributing

We appreciate all contributions to improve MMAction. Please refer to CONTRUBUTING.md for the contributing guideline.

Citation

If you use our codebase or models in your research, please cite this work. We will release a technical report later.

@misc{mmaction2019,
  author =       {Yue Zhao, Yuanjun Xiong, Dahua Lin},
  title =        {MMAction},
  howpublished = {\url{https://github.com/open-mmlab/mmaction}},
  year =         {2019}
}

Contact

If you have any question, please file an issue or contact the author:

Yue Zhao: [email protected]