PKU-YuanGroup/Helios

Helios: Real Real-Time Long Video Generation Model

⭐ 14B Real-Time Long Video Generation Model can be Cheaper, Faster but Keep Stronger than 1.3B ones ⭐

✨ Highlights

1. Without commonly used anti-drifting strategies (e.g., self-forcing, error-banks, keyframe sampling, or inverted sampling), Helios generates minute-scale videos with high quality and strong coherence.

2. Without standard acceleration techniques (e.g., KV-cache, causal masking, sparse/linear attention, TinyVAE, progressive noise schedules, hidden-state caching, or quantization), Helios achieves 19.5 FPS in end-to-end inference on a single H100 GPU.

3. We introduce optimizations that improve both training and inference throughput while reducing memory consumption, enabling image-diffusion-scale batch sizes during training while fitting up to four 14B models within 80 GB of GPU memory.

🎬 Video Demos

or you can click here to get the video. Some best prompts are here.

📣 Latest News!!

• [2026.07.28] 👋 CNAPS.AI has provided a flexible workflow of Helios on their online inference service platform. You can check their blog 1 and blog 2 for an easy tutorial and more adaptable parameters to better use Helios there.
• [2026.03.26] 🔥 Add summary of FAQ, Tips, and Tutorals: https://github.com/PKU-YuanGroup/Helios/issues/47.
• [2026.03.24] 👋 A community-made, unofficial YouTube tutorial for Helios is available here. It covers installation on a consumer-grade PC and supports 4K video generation.
• [2026.03.20] 🚀 Helios now supports Ahead-of-Time Compilation (AOTI) on Spaces, with special thanks to the HuggingFace Team! Please refer to this Space for a usage example.
• [2026.03.20] 🔧 Based on issue #38, we've identified several ways to further improve Helios's performance, such as fixing the i2v train-inference inconsistency and fully enabling Easy Anti-Drifting. Please refer to commits and correct.yaml for details.
• [2026.03.12] ⚡️ Please note that real-time generation performance depends not only on the GPU, but also on the CPU, memory, CUDA driver version, etc. As tested by a user on better hardware with single H100, Helios can reach up to 20.89 FPS!
• [2026.03.08] 🚀 Helios now fully supports Group Offloading and Context Parallelism! These features significantly optimize VRAM (only ~6GB) usage and enable inference across multiple GPUs with Ulysses Attention, Ring Attention, Unified Attention, and Ulysses Anything Attention.
• [2026.03.06] 👋 Cache-DiT now supports Helios, it offers Fully Cache Acceleration and Parallelism support for Helios! Special thanks to the Cache-DiT Team for their amazing work.
• [2026.03.06] 🔧 We fix the Parallel Inference logits for Helios, and provide an example here.
• [2026.03.06] 🚀 We official release the Gradio Demo, welcome to try it.
• [2026.03.05] 🔥 We are excited to announce the release of the Helios technical report on arXiv. We welcome discussions and feedback!
• [2026.03.04] 👋 Day-0 support for Ascend-NPU，with sincere gratitude to the Ascend Team for their support.
• [2026.03.04] 👋 Day-0 support for Diffusers，with special thanks to the HuggingFace Team for their support.
• [2026.03.04] 👋 Day-0 support for SGLang-Diffusion，with huge thanks to the SGLang Team for their support.
• [2026.03.04] 👋 Day-0 support for vLLM-Omni，with heartfelt gratitude to the vLLM Team for their support.
• [2026.03.04] 🔥 We've released the training/inference code and weights of Helios-Base, Helios-Mid and Helios-Distilled.

🔥 Friendly Links

If your work has improved Helios and you would like more people to see it, please inform us.

• Ascend-NPU: Developed by Huawei, this hardware is designed for efficient AI model training and inference, boosting performance in tasks like computer vision, natural language processing, and autonomous driving.
• Diffusers: A popular library designed for working with diffusion models and other generative models in deep learning. It supports easy integration and manipulation of a wide range of generative models.
• SGLang-Diffusion: An inference framework for accelerated image and video generation using diffusion models. It provides an end-to-end unified pipeline with optimized kernels and an efficient scheduler loop.
• vLLM-Omni: A fully disaggregated serving system for any-to-any models. vLLM-Omni breaks complex architectures into a stage-based graph, using a decoupled backend to maximize resource efficiency and throughput.
• Cache-DiT: A PyTorch-native and Flexible Inference Engine with Hybrid Cache Acceleration and Parallelism for DiTs. It built on top of the Diffusers library and now supports nearly ALL DiTs from Diffusers.
• CNAPS.AI: An AI creation platform for generating and editing visual content through text, images, and videos, with an accessible interface for creative production workflows.

⚙️ Requirements and Installation

Video Tutorial

If you prefer a step-by-step walkthrough, check out this community-made YouTube Tutorial. It covers local installation, 4K video generation, and how to run Helios on a consumer-grade PC, along with other practical usage tips.

Prepare Environment

# 0. Clone the repo
git clone --depth=1 https://github.com/PKU-YuanGroup/Helios.git
cd Helios
1. Create conda environment
conda create -n helios python=3.11.2
conda activate helios
2. Install PyTorch (adjust for your CUDA version)
CUDA 12.6
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu126
CUDA 12.8
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128
CUDA 13.0
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu130
3. Install dependencies
bash install.sh

Model Download

| Models | Download Link | Supports | Notes | |------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------|---------------------------------------------------------------------------------------------| | Helios-Base | 🤗 Huggingface 🤖 ModelScope | T2V ✅ I2V ✅ V2V ✅ Interactive ✅ | Best Quality, with v-prediction, standard CFG and custom HeliosScheduler. | | Helios-Mid | 🤗 Huggingface 🤖 ModelScope | T2V ✅ I2V ✅ V2V ✅ Interactive ✅ | Intermediate Ckpt, with v-prediction, CFG-Zero* and custom HeliosScheduler. | | Helios-Distilled | 🤗 Huggingface 🤖 ModelScope | T2V ✅ I2V ✅ V2V ✅ Interactive ✅ | Best Efficiency, with x0-prediction and custom HeliosDMDScheduler. |

💡Note:

* All three models share the same architecture, but Helios-Mid and Helios-Distilled use a more aggressive multi-scale sampling pipeline to achieve better efficiency.

* Helios-Mid is an intermediate checkpoint generated in the process of distilling Helios-Base into Helios-Distilled, and may not meet expected quality.

* For Image-to-Video or Video-to-Video, since training is based on Text-to-Video, these two functions may be slightly inferior to Text-to-Video. You may enable is_skip_first_chunk if you find the first few chunks are static or imporve the value of image_noise_sigma_min, image_noise_sigma_max, video_noise_sigma_min, and video_noise_sigma_max.

Download models using huggingface-cli: ``` sh pip install "huggingface_hub[cli]" huggingface-cli download BestWishYSH/Helios-Base --local-dir BestWishYSH/Helios-Base huggingface-cli download BestWishYSH/Helios-Mid --local-dir BestWishYSH/Helios-Mid huggingface-cli download BestWishYSH/Helios-Distilled --local-dir BestWishYSH/Helios-Distilled

Download models using modelscope-cli:

sh pip install modelscope modelscope download BestWishYSH/Helios-Base --local_dir BestWishYSH/Helios-Base modelscope download BestWishYSH/Helios-Mid --local_dir BestWishYSH/Helios-Mid modelscope download BestWishYSH/Helios-Distilled --local_dir BestWishYSH/Helios-Distilled

🚀 Inference

Helios uses an autoregressive approach that generates 33 frames per chunk. For optimal performance, num_frames should be set to a multiple of 33. If a non-multiple value is provided, it will be automatically rounded up to the nearest multiple of 33.
Example frame counts for different video lengths:
| num_frames | Adjusted Frames | 24 FPS | 16 FPS |
|------------|-----------------|--------|--------|
| 1449       | 1452 (33×44)    | ~60s (1min) | ~90s (1min 30s) |
| 720        | 726 (33×22)     | ~30s | ~45s |
| 240        | 264 (33×8)      | ~11s | ~16s |
| 129        | 132 (33×4)      | ~5.5s | ~8s |
| 81         | 99  (33×3)      | ~4s | ~6s |
Run the model
We provide inference scripts for all models covering text-to-video, image-to-video, and video-to-video in this directory.

bash cd scripts/inference

For Helios-Base

bash helios-base_t2v.sh bash helios-base_i2v.sh bash helios-base_v2v.sh

For Helios-Mid

bash helios-mid_t2v.sh bash helios-mid_i2v.sh bash helios-mid_v2v.sh

For Helios-Distilled

bash helios-distilled_t2v.sh bash helios-distilled_i2v.sh bash helios-distilled_v2v.sh

For Interactive

⚠️ This feature is still under development — results may not always meet expectations

cd scripts/inference/experiment_interactive

Sanity Check
Before trying your own inputs, we highly recommend going through the sanity check to find out if any hardware or software went wrong.
| Task    | Helios-Base                                                                                                            | Helios-Mid                                                                                                             | Helios-Distilled                                                                                                       |
| ------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| T2V |  |  |  |
| V2V |  |  |  |
✨ Group Offloading to Save VRAM
Helios supports group offloading to significantly reduce VRAM consumption, allowing you to run on GPU with limited memory footprint. For more details on the underlying mechanics, please refer to the documentation.
The Helios model below requires ~6GB of VRAM.

  
Click to expand the code

bash CUDA_VISIBLE_DEVICES=0 python infer_helios.py \ --base_model_path "BestWishYsh/Helios-Distilled" \ --transformer_path "BestWishYsh/Helios-Distilled" \ --sample_type "t2v" \ --prompt "A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and the vivid colors of its surroundings. A close-up shot with dynamic movement." \ --num_frames 240 \ --guidance_scale 1.0 \ --is_enable_stage2 \ --pyramid_num_inference_steps_list 2 2 2 \ --is_amplify_first_chunk \ --output_folder "./output_helios/helios-distilled" \ --enable_low_vram_mode \ --group_offloading_type "leaf_level"

  

✨ Context Parallelism on Multiple GPUs
Helios supports various Context Parallelism mechanisms, including Ulysses Attention, Ring Attention, Unified Attention, and Ulysses Anything Attention. For more details, please refer to the documentation.
For example, let's take Helios-Base with 4 GPUs.

  
Click to expand the code

bash CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node 4 infer_helios.py \ --enable_parallelism \ # remember to enable this config --cp_backend "ulysses" \ # ["ring", "ulysses", "unified", "ulysses_anything"] --base_model_path "BestWishYsh/Helios-Base" \ --transformer_path "BestWishYsh/Helios-Base" \ --sample_type "t2v" \ --num_frames 99 \ --fps 24 \ --prompt "A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and the vivid colors of its surroundings. A close-up shot with dynamic movement." \ --guidance_scale 5.0 \ --output_folder "./output_helios/helios-base"

  

✨ Diffusers Pipeline
Install diffusers from source:

bash pip install git+https://github.com/huggingface/diffusers.git

For example, let's take Helios-Distilled (Standard Pipeline).


  
Click to expand the code

bash import torch from diffusers import AutoModel, HeliosPyramidPipeline from diffusers.utils import export_to_video, load_video, load_image

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="vae", torch_dtype=torch.float32)

pipeline = HeliosPyramidPipeline.from_pretrained( "BestWishYsh/Helios-Distilled", vae=vae, torch_dtype=torch.bfloat16 ) pipeline.to("cuda")

negative_prompt = """ Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards """

# --- T2V --- prompt = """ A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and the vivid colors of its surroundings. A close-up shot with dynamic movement. """

output = pipeline( prompt=prompt, negative_prompt=negative_prompt, num_frames=240, pyramid_num_inference_steps_list=[2, 2, 2], guidance_scale=1.0, is_amplify_first_chunk=True, generator=torch.Generator("cuda").manual_seed(42), ).frames[0] export_to_video(output, "helios_distilled_t2v_output.mp4", fps=24)

# --- I2V --- i2v_prompt = """ A towering emerald wave surges forward, its crest curling with raw power and energy. Sunlight glints off the translucent water, illuminating the intricate textures and deep green hues within the wave’s body. A thick spray erupts from the breaking crest, casting a misty veil that dances above the churning surface. As the perspective widens, the immense scale of the wave becomes apparent, revealing the restless expanse of the ocean stretching beyond. The scene captures the ocean’s untamed beauty and relentless force, with every droplet and ripple shimmering in the light. The dynamic motion and vivid colors evoke both awe and respect for nature’s might. """ image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/wave.jpg"

output = pipeline( prompt=i2v_prompt, negative_prompt=negative_prompt, image=load_image(image_path).resize((640, 384)), num_frames=240, pyramid_num_inference_steps_list=[2, 2, 2], guidance_scale=1.0, is_amplify_first_chunk=True, generator=torch.Generator("cuda").manual_seed(42), ).frames[0] export_to_video(output, "helios_distilled_i2v_output.mp4", fps=24)

# --- V2V --- v2v_prompt = """ A bright yellow Lamborghini Huracn Tecnica speeds along a curving mountain road, surrounded by lush green trees under a partly cloudy sky. The car's sleek design and vibrant color stand out against the natural backdrop, emphasizing its dynamic movement. The road curves gently, with a guardrail visible on one side, adding depth to the scene. The motion blur captures the sense of speed and energy, creating a thrilling and exhilarating atmosphere. A front-facing shot from a slightly elevated angle, highlighting the car's aggressive stance and the surrounding greenery. """ video_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/car.mp4"

output = pipeline( prompt=v2v_prompt, negative_prompt=negative_prompt, video=load_video(video_path), num_frames=240, pyramid_num_inference_steps_list=[2, 2, 2], guidance_scale=1.0, is_amplify_first_chunk=True, generator=torch.Generator("cuda").manual_seed(42), ).frames[0] export_to_video(output, "helios_distilled_v2v_output.mp4", fps=24)

For example, let's take Helios-Distilled (Modular Pipeline).

  
Click to expand the code

bash import torch from diffusers import ModularPipeline, ClassifierFreeGuidance from diffusers.utils import export_to_video, load_image, load_video

mod_pipe = ModularPipeline.from_pretrained("BestWishYsh/Helios-Distilled") mod_pipe.load_components(torch_dtype=torch.bfloat16) mod_pipe.to("cuda")

# we need to upload guider to the model repo, so each checkpoint will be able to config their guidance d