Research map / Image generation / Editing and style transfer

EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation

CVPR 2025 · Image generation › Editing and style transfer · approach: Talking face generation · 14 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EchoMimicV2, an audio-driven half-body human animation framework that reduces control conditions via Audio-Pose Dynamic Harmonization, Head Partial Attention for headshot data augmentation, and a phase-specific denoising loss, plus a new half-body benchmark (EMTD).

Problem

Audio-driven human animation is largely limited to head regions, and existing half-body methods rely on many extra control conditions and complex injection modules that make training unstable and inference slow.

Approach

Progressively replace full-body pose conditions with hand-only pose while shifting audio control from lips to head to whole body (Audio-Pose Dynamic Harmonization), reuse head-shot data via masked cross-attention (Head Partial Attention), and apply loss terms specific to the early pose, middle detail, and late quality denoising phases (PhD Loss).

Key results

• EchoMimicV2 reaches FID 49.33 and FVD 598.45 on EMTD, versus 53.47 / 622.62 for MimicMotion and 58.98 / 1016.47 for AnimateAnyone.
• EchoMimicV2 attains the highest SSIM (0.738), PSNR (21.986), and CSIM (0.558) among the compared methods, and the best HKC (0.923) and HKV (25.28) hand metrics.
• Removing the phase-specific pose loss (w/o Lpose) raises FID to 51.30 and lowers SSIM to 0.695, and the straightforward baseline with no APDH or PhD Loss reaches FID 51.53.

Measured effects reported in the paper

Surprising findings

• Removing all pose conditions yields a fully audio-driven model that still captures general gestures correlated with audio tones, though it cannot produce specific gestures such as saluting.

Limitations

• Requires predefined hand pose sequences from human input; cannot generate hand poses directly from audio.
• Performance declines on non-cropped reference images such as full-body images.
• Evaluation benchmark EMTD is only partially released (qualitative results only, due to licensing), limiting reproducibility of quantitative comparisons.

Experimental setup

Datasets: HumanVid EchoMimic training data (headshot) internet half-body speaking videos

Benchmarks: EMTD

Metrics: FID FVD SSIM PSNR E-FID Sync-C Sync-D HKC HKV CSIM

Baselines: AnimateAnyone MimicMotion Vlogger CyberHost

Builds on: EchoMimic Latent Diffusion Model ReferenceNet AnimateAnyone

Closest papers by meaning

• EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions (AAAI 2025 · similarity 0.89)
• HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.89)
• 🎧MOSPA: Human Motion Generation Driven by Spatial Audio (NeurIPS 2025 · similarity 0.87)
• Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation (CVPR 2025 · similarity 0.86)
• Seamless Human Motion Composition with Blended Positional Encodings (CVPR 2024 · similarity 0.86)
• Free-viewpoint Human Animation with Pose-correlated Reference Selection (CVPR 2025 · similarity 0.86)