Tencent-Hunyuan/HunyuanVideo-1.5

★ 4,551⑂ 0

HunyuanVideo-1.5: A leading lightweight video generation model

4,551Star
0Fork
0Watch
0Issue
PythonLanguage
-License
Open on GitHub ↗
Created · last push · repository size 0 KB · default branch -

README

中文文档

HunyuanVideo-1.5

https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/HEAD/HunyuanVideo-1.5 Logo

🎬 HunyuanVideo-1.5: A leading lightweight video generation model

HunyuanVideo-1.5 is a video generation model that delivers top-tier quality with only 8.3B parameters, significantly lowering the barrier to usage. It runs smoothly on consumer-grade GPUs, making it accessible for every developer and creator. This repository provides the implementation and tools needed to generate creative videos.


👏 Join our WeChat and Discord | 💻 Official website Try our model!&nbsp&nbsp

🔥🔥🔥 News

🎥 Demo

🧩 Community Contributions

If you develop/use HunyuanVideo-1.5 in your projects, welcome to let us know.

📑 Open-source Plan

📋 Table of Contents

📖 Introduction

We present HunyuanVideo-1.5, a lightweight yet powerful video generation model that achieves state-of-the-art visual quality and motion coherence with only 8.3 billion parameters, enabling efficient inference on consumer-grade GPUs. This achievement is built upon several key components, including meticulous data curation, an advanced DiT architecture with selective and sliding tile attention(SSTA), enhanced bilingual understanding through glyph-aware text encoding, progressive pre-training and post-training, and an efficient video super-resolution network. Leveraging these designs, we developed a unified framework capable of high-quality text-to-video and image-to-video generation across multiple durations and resolutions. Extensive experiments demonstrate that this compact and proficient model establishes a new state-of-the-art among open-source models. By releasing the code and weights of HunyuanVideo-1.5, we provide the community with a high-performance foundation that significantly lowers the cost of video creation and research, making advanced video generation more accessible to all.

✨ Key Features

https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/HEAD/HunyuanVideo-1.5 DiT
https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/HEAD/HunyuanVideo-1.5 VSR

📜 System Requirements

Hardware Requirements

> Note: The memory requirements above are measured with model offloading enabled. If your GPU has sufficient memory, you may disable offloading for improved inference speed.

Software Requirements

🛠️ Dependencies and Installation

Step 1: Clone the Repository

git clone https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5.git
cd HunyuanVideo-1.5

Step 2: Install Basic Dependencies

pip install -r requirements.txt
pip install -i https://mirrors.tencent.com/pypi/simple/ --upgrade tencentcloud-sdk-python

Step 3: Install Attention Libraries

Install Flash Attention for faster inference and reduced GPU memory consumption. Detailed installation instructions are available at Flash Attention. flex-block-attn is only required for sparse attention to achieve faster inference and can be installed by the following command: 
  git clone https://github.com/Tencent-Hunyuan/flex-block-attn.git
  cd flex-block-attn
  git submodule update --init --recursive
  python3 setup.py install
To enable SageAttention for faster inference, you need to install it by the following command: > Note: Enabling SageAttention will automatically disable Flex-Block-Attention. 
  git clone https://github.com/cooper1637/SageAttention.git
  cd SageAttention 
  export EXT_PARALLEL=4 NVCC_APPEND_FLAGS="--threads 8" MAX_JOBS=32 # Optional
  python3 setup.py install
To enable fp8 gemm for transformer, you need to install it by the following command: 
  pip install sgl-kernel==0.3.18

🧱 Download Pretrained Models

💡 Distillation models and sparse attention models are still coming soon. Please stay tuned for the latest updates on the Hugging Face Model Card.

Download the pretrained models before generating videos. Detailed instructions are available at checkpoints-download.md.

Model Cards

|ModelName| Download | |-|---------------------------| |HunyuanVideo-1.5-480P-T2V|480P-T2V | |HunyuanVideo-1.5-480P-I2V |480P-I2V | |HunyuanVideo-1.5-480P-T2V-cfg-distill | 480P-T2V-cfg-distill | |HunyuanVideo-1.5-480P-I2V-cfg-distill |480P-I2V-cfg-distill | |HunyuanVideo-1.5-480P-I2V-step-distill |480P-I2V-step-distill | |HunyuanVideo-1.5-720P-T2V|720P-T2V | |HunyuanVideo-1.5-720P-I2V |720P-I2V | |HunyuanVideo-1.5-720P-T2V-cfg-distill| Coming soon | |HunyuanVideo-1.5-720P-I2V-cfg-distill |720P-I2V-cfg-distill | |HunyuanVideo-1.5-720P-T2V-sparse-cfg-distill| Coming soon | |HunyuanVideo-1.5-720P-I2V-sparse-cfg-distill |720P-I2V-sparse-cfg-distill | |HunyuanVideo-1.5-720P-sr-step-distill |720P-sr | |HunyuanVideo-1.5-1080P-sr-step-distill |1080P-sr |

📝 Prompt Guide

Prompt Writing Handbook

Prompt enhancement plays a crucial role in enabling our model to generate high-quality videos. By writing longer and more detailed prompts, the generated video will be significantly improved. We encourage you to craft comprehensive and descriptive prompts to achieve the best possible video quality. We recommend community partners consulting our official guide on how to write effective prompts.

Reference: HunyuanVideo-1.5 Prompt Handbook

System Prompts for Automatic Prompt Enhancement

For users seeking to optimize prompts for other large models, it is recommended to consult the definition of t2v_rewrite_system_prompt in the file hyvideo/utils/rewrite/t2v_prompt.py to guide text-to-video rewriting. Similarly, for image-to-video rewriting, refer to the definition of i2v_rewrite_system_prompt in hyvideo/utils/rewrite/i2v_prompt.py.

🔑 Inference

Inference with Source Code

For prompt rewriting, we recommend using Gemini or models deployed via vLLM. This codebase currently only supports models compatible with the vLLM API. If you wish to use Gemini, you will need to implement your own interface calls.

For models with a vLLM API, note that T2V (text-to-video) and I2V (image-to-video) have different recommended models and environment variables:

You may set the above model names to any other vLLM-compatible models you have deployed (including HuggingFace models).
Rewriting is enabled by default (--rewrite defaults to true); to disable it explicitly, use --rewrite false or --rewrite 0. If no vLLM endpoint is configured, the pipeline runs without remote rewriting.

Example: Generate a video (works for both T2V and I2V; set IMAGE_PATH=none for T2V or provide an image path for I2V)

💡 Tip: For faster inference speed, you can enable the step-distilled model using the --enable_step_distill parameter. The step-distilled model (480p I2V) can generate videos in 8 or 12 steps (recommended), achieving up to 75% speedup on RTX 4090 while maintaining comparable quality.
> Tips: If your GPU memory is > 14GB but you encounter OOM (Out of Memory) errors during generation, you can try setting the following environment variable before running:
> export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:128
Tips: If you have limited CPU memory and encounter OOM during inference, you can try disable overlapped group offloading by adding the following argument:
> --overlap_group_offloading false
export T2V_REWRITE_BASE_URL="<your_vllm_server_base_url>"
export T2V_REWRITE_MODEL_NAME="<your_model_name>"
export I2V_REWRITE_BASE_URL="<your_vllm_server_base_url>"
export I2V_REWRITE_MODEL_NAME="<your_model_name>"

PROMPT='A girl holding a paper with words "Hello, world!"'


IMAGE_PATH=/path/to/image.png # Optional, none or  to enable i2v mode
SEED=1
ASPECT_RATIO=16:9
RESOLUTION=480p
OUTPUT_PATH=./outputs/output.mp4
MODEL_PATH=./ckpts # Path to pretrained model


Configuration for faster inference

N_INFERENCE_GPU=8 # Parallel inference GPU count
CFG_DISTILLED=true # Inference with CFG distilled model, 2x speedup
SAGE_ATTN=true # Inference with SageAttention
SPARSE_ATTN=false # Inference with sparse attention (only 720p models are equipped with sparse attention). Please ensure flex-block-attn is installed
OVERLAP_GROUP_OFFLOADING=true # Only valid when group offloading is enabled, significantly increases CPU memory usage but speeds up inference
ENABLE_CACHE=true # Enable feature cache during inference. Significantly speeds up inference.
CACHE_TYPE=deepcache # Support: deepcache, teacache, taylorcache
ENABLE_STEP_DISTILL=true # Enable step distilled model for 480p I2V, recommended 8 or 12 steps, up to 6x speedup

Configuration for better quality

REWRITE=true # Enable prompt rewriting. Please ensure rewrite vLLM server is deployed and configured.
ENABLE_SR=true # Enable super resolution

torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \
  --prompt "$PROMPT" \
  --image_path $IMAGE_PATH \
  --resolution $RESOLUTION \
  --aspect_ratio $ASPECT_RATIO \
  --seed $SEED \
  --rewrite $REWRITE \
  --cfg_distilled $CFG_DISTILLED \
  --enable_step_distill $ENABLE_STEP_DISTILL \
  --sparse_attn $SPARSE_ATTN --use_sageattn $SAGE_ATTN \
  --enable_cache $ENABLE_CACHE --cache_type $CACHE_TYPE \
  --overlap_group_offloading $OVERLAP_GROUP_OFFLOADING \
  --sr $ENABLE_SR --save_pre_sr_video \
  --output_path $OUTPUT_PATH \
  --model_path $MODEL_PATH

Command Line Arguments

| Argument | Type | Required | Default | Description | |----------|------|----------|---------|-------------| | --prompt | str | Yes | - | Text prompt for video generation | | --negative_prompt | str | No | '' | Negative prompt for video generation | | --resolution | str | Yes | - | Video resolution: 480p or 720p | | --model_path | str | Yes | - | Path to pretrained model directory | | --aspect_ratio | str | No | 16:9 | Aspect ratio of the output video | | --num_inference_steps | int | No | 50 | Number of inference steps | | --video_length | int | No | 121 | Number of frames to generate | | --seed | int | No | 123 | Random seed for reproducibility | | --image_path | str | No | None | Path to reference image (enables i2v mode). Use none or None to explicitly use text-to-video mode | | --output_path | str | No | None | Output file path (if not provided, saves to ./outputs/output_{transformer_version}_{timestamp}.mp4) | | --sr | bool | No | true | Enable super resolution (use --sr false or --sr 0 to disable) | | --save_pre_sr_video | bool | No | false | Save original video before super resolution (use --save_pre_sr_video or --save_pre_sr_video true to enable, only effective when super resolution is enabled) | | --rewrite | bool | No | true | Enable prompt rewriting (use --rewrite false or --rewrite 0 to disable, may result in lower quality video generation) | | --cfg_distilled | bool | No | false | Enable CFG distilled model for faster inference (~2x speedup, use --cfg_distilled or --cfg_distilled true to enable) | | --enable_step_distill | bool | No | false | Enable step distilled model for 480p I2V (recommended 8 or 12 steps, ~75% speedup on RTX 4090, use --enable_step_distill or --enable_step_distill true to enable) | | --sparse_attn | bool | No | false | Enable sparse attention for faster inference (~1.5-2x speedup, requires H-series GPUs, auto-enables CFG distilled, use --sparse_attn or --sparse_attn true to enable) | | --offloading | bool | No | true | Enable CPU offloading (use --offloading false or --offloading 0 to disable for faster inference if GPU memory allows) | | --group_offloading | bool | No | None | Enable group offloading (default: None, automatically enabled if offloading is enabled. Use --group_offloading or --group_offloading true/1 to enable, --group_offloading false/0 to disable) | | --overlap_group_offloading | bool | No | true | Enable overlap group offloading (default: true). Significantly increases CPU memory usage but speeds up inference. Use --overlap_group_offloading or --overlap_group_offloading true/1 to enable, --overlap_group_offloading false/0 to disable | | --dtype | str | No | bf16 | Data type for transformer: bf16 (faster, lower memory) or fp32 (better quality, slower, higher memory) | | --use_sageattn | bool | No | false | Enable SageAttention (use --use_sageattn or --use_sageattn true/1 to enable, --use_sageattn false/0 to disable) | | --sage_blocks_range | str | No | 0-53 | SageAttention blocks range (e.g., 0-5 or 0,1,2,3,4,5) | | --enable_cache | bool | No | false | Enable cache for transformer (use --enable_cache or --enable_cache true/1 to enable, --enable_cache false/0 to disable) | | --cache_type | str | No | deepcache | Cache type for transformer (e.g., deepcache, teacache, taylorcache) | | --no_cache_block_id | str | No | 53 | Blocks to exclude from deepcache (e.g., 0-5 or 0,1,2,3,4,5) | | --cache_start_step | int | No | 11 | Start step to skip when using cache | | --cache_end_step | int | No | 45 | End step to skip when using cache | | --total_steps | int | No | 50 | Total inference steps | | --cache_step_interval | int |

More Video Trending projects

1

Genymobile / scrcpy

C★ 149,650⑂ 0
→
2

harry0703 / MoneyPrinterTurbo

Python★ 123,776⑂ 0
→
3

obsproject / obs-studio

C★ 76,223⑂ 0
→
4

FFmpeg / FFmpeg

C★ 64,235⑂ 0
→
5

calesthio / OpenMontage

Python★ 59,205⑂ 0
→
6

heygen-com / hyperframes

TypeScript★ 50,163⑂ 0
→