Research map / Image generation / Editing and style transfer
Talking face generation
Generate emotional, expressive talking-head video from audio and style signals.
21 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2025) · an approach in Editing and style transfer, keyword cluster “face · facial · identity”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Leveling off Several papers from 2025–26 still report on 1 tracked benchmark, but the best result has barely moved since 2024.
10 papers in 2023–24 and 6 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 3.7% to 1.5%.
Open questions the data points to
- Papers report EAT on MEAD (FID) anywhere from 19.84 to 32.7: comparisons against it depend on whose number you take.
- Papers report Hallo on HDTF (FVD) anywhere from 343.02 to 501.07: comparisons against it depend on whose number you take.
What looks settled
- HDTF (FID): best up to 2024 15.16, best in 2025–26 18.98 from 3 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · A Face-to-Face Neural Conversation Model (CVPR 2018 · 17 citations)
- 2019 · Face Reconstruction from Voice using Generative Adversarial Networks (NeurIPS 2019 · 45 citations)
- 2020 · FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis (AAAI 2020 · 48 citations)
- 2022 · Talking Face Generation With Multilingual TTS (CVPR 2022 · 17 citations)
- 2023 · StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-Based Generator (CVPR 2023 · 64 citations)
- 2024 · Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style (AAAI 2024 · 28 citations)
- 2025 · EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions (AAAI 2025 · 54 citations)
Earliest papers here
- A Face-to-Face Neural Conversation Model (CVPR 2018 · 17 citations)
- Face Reconstruction from Voice using Generative Adversarial Networks (NeurIPS 2019 · 45 citations)
- FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis (AAAI 2020 · 48 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
HDTF · FID (lower is better)
5 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 15.16 | FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization (CVPR 2024 · 26 citations) |
MEAD · FID (lower is better)
4 papers, 2023–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 15.89 | High-Fidelity Generalized Emotional Talking Face Generation With Multi-Modal Emotion Space Learning (CVPR 2023 · 37 citations) |
What has been tried
21 papers here report 202 measured changes (a component added, removed or swapped, compared with a baseline): helped 83%, no clear effect 2%, hurt 13%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use landmark-only instead of audio-only driving | HDTF · FID | 29.136 -> 22.970 | Helped | EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions (AAAI 2025 · 54 citations) |
| Add emotion text and the CLIP text encoder | MEAD · F-LMD | 2.913 -> 2.696 | Helped | Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style (AAAI 2024 · 28 citations) |
| Replace SMM with GMM in ExpFlow | MEAD · expression diversity and robustness | Hurt | FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization (CVPR 2024 · 26 citations) | |
| Add Initial Pose phase | EMTD · FID | 49.99 -> 49.33 | Helped | EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation (CVPR 2025 · 14 citations) |
| Add blendshape augmentation | TEAD test set · MSE | 0.051 -> 0.012 | Helped | ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment (AAAI 2024 · 14 citations) |
| Use Latent ControlNet instead of pixel-space ControlNet | CelebA-HQ hair transfer · CLIP-I | 0.423 -> 0.434 | Helped | Stable-Hair: Real-World Hair Transfer via Diffusion Model (AAAI 2025 · 13 citations) |
| Predict a residual Δ over body pose | MENTOR · FID | 52.27 -> 15.36 | Helped | VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis (CVPR 2025 · 13 citations) |
| add Audio Aware Aggregation rather than slot attention | Ablation study · PSNR | 31.25 -> 32.64 | Helped | AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis (AAAI 2024 · 12 citations) |
| Pretrain ASCCL with paired data rather than use it untrained | MEAD Inter-ID · FAD | 1.509 -> 1.234 | Helped | Learning Adaptive Spatial Coherent Correlations for Speech-Preserving Facial Expression Manipulation (CVPR 2024 · 10 citations) |
| Add the Audio-text Responsive Interaction component | ViCo Dtest and Dood · RTLCC | 0.206 -> 0.165 | Helped | CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation (CVPR 2024 · 6 citations) |
| add Mixture of Emotion Experts | DH-FaceEmoVid-150 · FID | 58.411 -> 39.619 | Helped | MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animation (CVPR 2025 · 5 citations) |
| Add IAQ-CL instead of static facial feature extraction | LRS3 test, audio input · UTMOS | 2.945 -> 3.227 | Helped | Seeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice Conversion (AAAI 2025 · 2 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| EAT | MEAD | FID | 19.84 – 32.7 | 2 papers, e.g. CVPR 2024 and CVPR 2025 |
| Hallo | HDTF | FVD | 343.02 – 501.07 | 2 papers, e.g. CVPR 2025 and AAAI 2025 |
| Hallo | HDTF | FID | 28.61 – 37.66 | 2 papers, e.g. CVPR 2025 and AAAI 2025 |
What reviewers will expect
Datasets and baselines named in the 6 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HDTF · 3 of 6 papers
- CelebV-HQ · 2 of 6 papers
Baselines
- SadTalker · 2 of 6 papers
- AniPortrait · 2 of 6 papers
- Hallo · 2 of 6 papers
- StyleTalk · 2 of 6 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 16 papers; from OpenAlex and the papers' affiliations).
- Shanghai Jiao Tong University · 4 papers
- Ant Group · 3 papers
- Alibaba Group · 2 papers
- University of Chinese Academy of Sciences · 2 papers
- Tencent · 2 papers
- Hong Kong University of Science and Technology · 2 papers
- Zhejiang University · 2 papers
Use it
Download BibTeX of 21 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Identity-preserving face manipulation (88 papers)
- Diffusion face generation and editing (61 papers)
Elsewhere in the topic
- Instruction-based diffusion editing (151 papers)
- Texture and sketch synthesis (130 papers)
- Image-to-image translation (127 papers)
- Arbitrary neural style transfer (124 papers)
- Inversion-based diffusion editing (113 papers)
- Localized and attribute-specific editing (73 papers)
- Virtual try-on (46 papers)
- Pose-guided person image synthesis (29 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).