antgroup/echomimic_v2

★ 4,653⑂ 0

[CVPR 2025] EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation

4,653Star
0Fork
0Watch
0Issue
PythonLanguage
-License
Open on GitHub ↗
Created · last push · repository size 0 KB · default branch -

README

EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation

Rang Meng1  Xingyu Zhang  Yuming Li2  Chenguang Ma2
Terminal Technology Department, Alipay, Ant Group.

1Core Contributor  2Corresponding Authors

🚀 EchoMimic Series

📣 Updates

🌅 Gallery

Introduction

English Driven Audio

Chinese Driven Audio

⚒️ Automatic Installation

Download the Codes

```bash git clone https://github.com/antgroup/echomimic_v2 cd echomimic_v2 ```

Automatic Setup

```bash sh linux_setup.sh ```

⚒️ Manual Installation

Download the Codes

```bash git clone https://github.com/antgroup/echomimic_v2 cd echomimic_v2 ```

Python Environment Setup

Create conda environment (Recommended): ```bash conda create -n echomimic python=3.10 conda activate echomimic ``` Install packages with pip ```bash pip install pip -U pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 xformers==0.0.28.post3 --index-url https://download.pytorch.org/whl/cu124 pip install torchao --index-url https://download.pytorch.org/whl/nightly/cu124 pip install -r requirements.txt pip install --no-deps facenet_pytorch==2.6.0 ```

Download ffmpeg-static

Download and decompress ffmpeg-static, then ``` export FFMPEG_PATH=/path/to/ffmpeg-4.4-amd64-static ```

Download pretrained weights

```shell git lfs install git clone https://huggingface.co/BadToBest/EchoMimicV2 pretrained_weights ``` The pretrained_weights is organized as follows. ``` ./pretrained_weights/ ├── denoising_unet.pth ├── reference_unet.pth ├── motion_module.pth ├── pose_encoder.pth ├── sd-vae-ft-mse │ └── ... └── audio_processor └── tiny.pt ``` In which denoising_unet.pth / reference_unet.pth / motion_module.pth / pose_encoder.pth are the main checkpoints of EchoMimic. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:

Inference on Demo

Run the gradio: ```bash python app.py ``` Run the python inference script: ```bash python infer.py --config='./configs/prompts/infer.yaml' ``` Run the python inference script for accelerated version. Make sure to check out the configuration for accelerated inference: ```bash python infer_acc.py --config='./configs/prompts/infer_acc.yaml' ```

EMTD Dataset

Download dataset: ```bash python ./EMTD_dataset/download.py ``` Slice dataset: ```bash bash ./EMTD_dataset/slice.sh ``` Process dataset: ```bash python ./EMTD_dataset/preprocess.py ``` Make sure to check out the discussions to learn how to start the inference.

📝 Release Plans

| Status | Milestone | ETA | |:--------:|:-------------------------------------------------------------------------|:--:| | ✅ | The inference source code of EchoMimicV2 meet everyone on GitHub | 21st Nov, 2024 | | ✅ | Pretrained models trained on English and Mandarin Chinese on HuggingFace | 21st Nov, 2024 | | ✅ | Pretrained models trained on English and Mandarin Chinese on ModelScope | 21st Nov, 2024 | | ✅ | EMTD dataset list and processing scripts | 21st Nov, 2024 | | ✅ | Jupyter demo with pose and reference image alignmnet | 16st Dec, 2024 | | ✅ | Accelerated models | 3st Jan, 2025 | | 🚀 | Online Demo on ModelScope to be released | TBD | | 🚀 | Online Demo on HuggingFace to be released | TBD |

⚖️ Disclaimer

This project is intended for academic research, and we explicitly disclaim any responsibility for user-generated content. Users are solely liable for their actions while using the generative model. The project contributors have no legal affiliation with, nor accountability for, users' behaviors. It is imperative to use the generative model responsibly, adhering to both ethical and legal standards.

🙏🏻 Acknowledgements

We would like to thank the contributors to the MimicMotion and Moore-AnimateAnyone repositories, for their open research and exploration. We are also grateful to CyberHost and Vlogger for their outstanding work in the area of audio-driven human animation. If we missed any open-source projects or related articles, we would like to complement the acknowledgement of this specific work immediately.

📒 Citation

If you find our work useful for your research, please consider citing the paper : ``` @article{meng2024echomimicv2, title={EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation}, author={Meng, Rang and Zhang, Xingyu and Li, Yuming and Ma, Chenguang}, journal={arXiv preprint arXiv:2411.10061}, year={2024} } @article{meng2025echomimicv3, title={Echomimicv3: 1.3 b parameters are all you need for unified multi-modal and multi-task human animation}, author={Meng, Rang and Wang, Yan and Wu, Weipeng and Zheng, Ruobing and Li, Yuming and Ma, Chenguang}, journal={arXiv preprint arXiv:2507.03905}, year={2025} } @article{meng2026echotorrent, title={EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation}, author={Meng, Rang and Wu, Weipeng and Yin, Yingjie and Li, Yuming and Ma, Chenguang}, journal={arXiv preprint arXiv:2602.13669}, year={2026} } ```

🌟 Star History

Star History Chart

More Video Trending projects

1

Genymobile / scrcpy

C★ 149,650⑂ 0
→
2

harry0703 / MoneyPrinterTurbo

Python★ 123,776⑂ 0
→
3

obsproject / obs-studio

C★ 76,223⑂ 0
→
4

FFmpeg / FFmpeg

C★ 64,235⑂ 0
→
5

calesthio / OpenMontage

Python★ 59,205⑂ 0
→
6

heygen-com / hyperframes

TypeScript★ 50,163⑂ 0
→