Research map / Video / Video generation
FacEDiT: Talking Head Video Editing via Facial Motion Infilling
Sung-Bin Kim, Joohyun Chang, David Harwath, Tae-Hyun Oh
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FacEDiT, a method for local talking head video editing (insertion, deletion, substitution) that synthesizes replacement segments via facial motion infilling while preserving identity, unedited regions, and boundary continuity.
Problem
Editing a local segment of a talking head video without reshooting the whole scene is challenging and underexplored; it requires variable-duration local rewriting while preserving identity, unedited regions, and boundary continuity.
Approach
Frames talking head editing as facial motion infilling: a replacement segment of variable duration is synthesized to fill the edited span while the rest of the video is left unchanged, enforcing boundary continuity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace early fusion with cross-attention speech conditioning (ablation) | FacEDiT · FacEDiTBench editing | LSE-D: 7.292 -> 7.179 | Improved |
| use WavLM instead of Wav2Vec (ablation) | FacEDiT · FacEDiTBench editing | LSE-D: 7.223 -> 7.179 | Improved |
| add local temporal attention bias (ablation) | FacEDiT · FacEDiTBench editing | M_continuity: 1.097 -> 0.829 | Improved |
| add local temporal attention bias (ablation) | FacEDiT · FacEDiTBench editing | FVD: 65.559 -> 61.955 | Improved |
| add temporal smoothness loss (ablation) | FacEDiT · FacEDiTBench editing | M_continuity: 0.829 -> 0.800 | Improved |
| add temporal smoothness loss (ablation) | FacEDiT · FacEDiTBench RMS acceleration | RMS acceleration: 0.001924 -> 0.001671 | Improved |
| mask-infilling model vs adapted portrait animation baselines | FacEDiT · FacEDiTBench editing | P_continuity: 2.420 vs 7.242 (Hallo) | Improved |
| longer edit durations (10-20 s to >=30 s) | FacEDiT · long edits | LSE-C: 6.061 -> 5.178 | Hurt |
| longer edit durations (10-20 s to >=30 s) | FacEDiT · long edits | M_continuity: 0.360 -> 0.549 | Hurt |
| same model with future-motion mask for portrait animation | FacEDiT · HDTF portrait animation | FVD: 31.662 vs 38.643 (Hallo3) | Improved |
| same model with lip-channel mask for lip sync | FacEDiT · HDTF lip synchronization | FVD: 29.381 vs 23.044 (LatentSync) | No clear effect |
| full-sequence generation by baselines instead of local stitching | portrait animation baselines · FacEDiTBench editing | IDSIM: 0.971 (ours) vs 0.756-0.847 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Hallo3 — Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer (CVPR 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation (CVPR 2025 · similarity 0.90)
- Occlusion-Insensitive Talking Head Video Generation via Facelet Compensation (AAAI 2025 · similarity 0.89)
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.87)
- Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model (ICML 2025 · similarity 0.87)
- SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis (CVPR 2024 · similarity 0.87)
- AnyTalk: Multi-modal Driven Multi-domain Talking Head Generation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
