Research map / Video / Video generation
Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
Yubo Huang, Hailong Guo, Fangtai Wu, Weiqiang Wang, Shijie Huang, Qijun Gan, Shifeng Zhang, Lin Liu, Sirui Zhao, Enhong Chen, Jiaming Liu, Steven Hoi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| store noisy instead of clean history in the KV cache (History Corrupt) (ablation) | Live Avatar · GenBench-LongVideo | ASE / Dino-S: ASE 2.90 -> 3.42; Dino-S 0.81 -> 0.97 | Improved |
| add Adaptive Attention Sink (ablation) | Live Avatar · GenBench-LongVideo | ASE: 3.13 -> 3.42 | Improved |
| add Rolling RoPE (ablation) | Live Avatar · GenBench-LongVideo | Dino-S: 0.86 -> 0.97 | Improved |
| share the noise level between KV cache and current block (timestep-forcing) vs fixed-noise cache (ablation) | Live Avatar · GenBench-LongVideo | T.Flicker: 0.891 -> 0.971 (clean cache 0.876) | Improved |
| add Timestep-forcing Pipeline Parallelism on top of DMD (4 GPUs) (ablation) | Live Avatar · inference efficiency | FPS: 4.50 (SP4) -> 10.16 | Improved |
| apply DMD distillation (80 -> 5 NFE) (ablation) | Live Avatar · inference efficiency | FPS: 0.29 -> 3.66 | Improved |
| add sequence parallelism across 4 GPUs (ablation) | Live Avatar · inference efficiency | FPS: 3.66 -> 4.50 | No clear effect |
| add kernel-level optimizations (FP8, FA3, compile) (ablation) | Live Avatar · inference efficiency | FPS / TTFF: FPS 20.88 -> 45.2; TTFF 2.89 -> 1.21 | Improved |
| use 4 denoising steps instead of 2 (ablation) | Live Avatar · GenBench-ShortVideo | Sync-C: 6.41 -> 7.03 | Improved |
| reduce block size from 3 latents to 1 (ablation) | Live Avatar · GenBench-ShortVideo | Sync-C: 7.03 -> 4.12 | Hurt |
| increase block size from 3 to 4 latents (ablation) | Live Avatar · GenBench-ShortVideo | Sync-C / TTFF: Sync-C 7.03 -> 6.98; TTFF 0.94 -> 1.21 | No clear effect |
| quantize to FP8 (ablation) | Live Avatar · GenBench-Short | FPS / IQA: FPS 36.0 -> 45.2; IQA 4.55 -> 4.51 | Mixed |
Closest papers by meaning
- DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking head Video Generation (ICLR 2025 · similarity 0.88)
- FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation (CVPR 2025 · similarity 0.87)
- Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation (CVPR 2025 · similarity 0.87)
- GAIA: Zero-shot Talking Avatar Generation (ICLR 2024 · similarity 0.86)
- RAM-Avatar: Real-time Photo-Realistic Avatar from Monocular Videos with Full-body Control (CVPR 2024 · similarity 0.86)
- VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
