novitalabs/animateanyone

Unofficial Implementation of Animate Anyone by Novita AI

Animate Anyone

Overview

This repository currently provides the unofficial pre-trained weights and inference code of Animate Anyone. It is inspired by the implementation of the MooreThreads/Moore-AnimateAnyone repository and we made some adjustments to the training process and datasets.

Samples

demo1.mp4

demo4.mp4

demo2.mp4

demo3.mp4

Quickstart

Build Environtment

We Recommend a python version >=3.10 and cuda version =11.7. Then build environment as follows:

# [Optional] Create a virtual env
python -m venv .venv
source .venv/bin/activate
# Install with pip:
pip install -r requirements.txt

Download weights

Automatically downloading: You can run the following command to download weights automatically:

python tools/download_weights.py

Weights will be placed under the ./pretrained_weights direcotry. The whole downloading process may take a long time.

Inference

Here is the cli command for running inference scripts:

python -m scripts.pose2vid --config ./configs/prompts/animation.yaml -W 512 -H 784 -L 64

You can refer the format of animation.yaml to add your own reference images or pose videos. To convert the raw video into a pose video (keypoint sequence), you can run with the following command:

python tools/vid2pose.py --video_path /path/to/your/video.mp4

Or try it on Novita AI

We've deployed this model on Novita AI, and you can try it out with Playground ➡️ https://novita.ai/playground#animate-anyone .

Acknowledgements

This project is based on MooreThreads/Moore-AnimateAnyone which is licensed under the Apache License 2.0. We thank to the authors of Animate Anyone and MooreThreads/Moore-AnimateAnyone, for their open research and exploration.

animateanyone's Issues

更详细的安装方法

可以提供一个更简单的安装方式吗

ModuleNotFoundError: No module named 'src.dataset'

In pipeline_pose2vid_long.py, trying to load a module called src.dataset.utils but the source code doesn't exist.
Did you forget to upload it?

The following is the full text of the executed command and the error output.

python -m scripts.pose2vid --config ./configs/prompts/animation.yaml -W 512 -H 784 -L 64
/home/ryosuke/.pyenv/versions/animate-anyone/lib/python3.10/site-packages/diffusers/models/dual_transformer_2d.py:20: FutureWarning: `DualTransformer2DModel` is deprecated and will be removed in version 0.29. Importing `DualTransformer2DModel` from `diffusers.models.dual_transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.transformers.dual_transformer_2d import DualTransformer2DModel`, instead.
  deprecate("DualTransformer2DModel", "0.29", deprecation_message)
Traceback (most recent call last):
  File "/home/ryosuke/.pyenv/versions/3.10.11/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/ryosuke/.pyenv/versions/3.10.11/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/media/NAS/users/ryosuke/projects/dance/AnimateAnyone/scripts/pose2vid.py", line 23, in <module>
    from src.pipelines.pipeline_pose2vid_long import Pose2VideoPipeline
  File "/media/NAS/users/ryosuke/projects/dance/AnimateAnyone/src/pipelines/pipeline_pose2vid_long.py", line 28, in <module>
    from ..dataset.utils import SquarePad
ModuleNotFoundError: No module named 'src.dataset'

环境问题

所提供的requirements.txt里，torch=2.1.1和xformers=0.0.22是冲突的。

而且，diffusers的import也报了以下错误：

ImportError: cannot import name 'GLIGENTextBoundingboxProjection' from 'diffusers.models.embeddings'

想学习AnimateAnyone快加学习交流群

pip install error

No such file or directory: './configs/inference/ref_images/anyone-3.png'

Would you mind share your train adjustments

Amazing work, it's better than many open source animate anyone. Would mind share your train details, Thank you

python tools/download_weights.py file not found

Will you share model weights?