Research map / Image generation / Text-to-image diffusion
FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation
Mengchao Wang, Wang Qiang, Fan Jiang, Mu Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add timestep-wise gating on top of layer-wise fusion (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | Sync-C: 5.509 -> 5.704 | Improved |
| add timestep-wise gating on top of layer-wise fusion (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | FVD: 357.963 -> 341.181 | Improved |
| use layer-wise fusion instead of expert-level fusion (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | Sync-C: 5.565 -> 5.704 | Improved |
| use layer-wise fusion instead of module-level fusion (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | Sync-C: 5.649 -> 5.704 | Improved |
| use decoupled multi-expert TLPO instead of single-LoRA DPO (ablation) | FantasyTalking · Audio-driven portrait animation benchmark ablation | Sync-C: 4.381 -> 5.704 | Improved |
| use TLPO instead of IPO (ablation) | FantasyTalking · Audio-driven portrait animation benchmark ablation | Sync-C: 4.375 -> 5.704 | Improved |
| use TLPO instead of SimPO (ablation) | FantasyTalking · Audio-driven portrait animation benchmark ablation | Sync-C: 4.546 -> 5.704 | Improved |
| raise expert LoRA rank from 32 to 128 (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | Sync-C: 5.546 -> 5.704 | Improved |
| raise expert LoRA rank from 128 to 256 (ablation) | TLPO · Audio-driven portrait animation benchmark ablation | FVD: 341.181 -> 341.174 | No clear effect |
| apply TLPO to FantasyTalking | FantasyTalking · Audio-driven portrait animation benchmark ablation | Sync-C: 3.154 -> 5.704 | Improved |
| apply TLPO to FantasyTalking | FantasyTalking · Audio-driven portrait animation benchmark ablation | FVD: 483.108 -> 341.181 | Improved |
| fine-tune Qwen2.5-Omni on multidimensional preference data (Talking-Critic) | Qwen2.5-Omni · Reward test set (1K human-annotated) | LS preference accuracy: 52.63 -> 86.94 | Improved |
Closest papers by meaning
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.88)
- Sonic: Shifting Focus to Global Audio Perception in Portrait Animation (CVPR 2025 · similarity 0.88)
- FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portrait (ICCV 2025 · similarity 0.88)
- SPRO: Improving Image Generation via Self-Play (NeurIPS 2025 · similarity 0.88)
- Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation (CVPR 2025 · similarity 0.88)
- DreamAlign: Dynamic Text-to-3D Optimization with Human Preference Alignment (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
