Research map / Video / Video generation
Text-to-motion diffusion
Synthesize human motion from text using diffusion models.
140 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Video generation, keyword cluster “audio · editing · talking”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 4 of 6 tracked benchmarks.
43 papers in 2023–24 and 95 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 18.9% to 8.5%.
Open questions the data points to
- HumanML3D (FID): best went from 0.084 (up to 2024) to 0.032 in 2025 — LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning (ICLR 2025 · 0 citations)
- HumanML3D (R-Precision Top-1): best went from 0.513 (up to 2024) to 0.557 in 2025 — LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning (ICLR 2025 · 0 citations)
- KIT-ML (FID): best went from 0.367 (up to 2024) to 0.14 in 2025 — StickMotion: Generating 3D Human Motions by Drawing a Stickman (CVPR 2025 · 4 citations)
- AIST++ (FIDk): best went from 17.25 (up to 2024) to 15.7 in 2025 — Controllable 3D Dance Generation Using Diffusion-Based Transformer U-Net (AAAI 2025 · 3 citations)
- Papers report MoMask on HumanML3D (FID) anywhere from 0.045 to 0.116: comparisons against it depend on whose number you take.
- Papers report MoMask on KIT-ML (FID) anywhere from 0.204 to 0.411: comparisons against it depend on whose number you take.
No longer reported by papers here: HumanML3D test (FID, last in 2024); HumanML3D test (R Precision (top 3), last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions (AAAI 2020 · 48 citations)
- 2022 · You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection (NeurIPS 2022 · 6 citations)
- 2023 · Executing Your Commands via Motion Diffusion in Latent Space (CVPR 2023 · 341 citations)
- 2024 · DisCo: Disentangled Control for Realistic Human Dance Generation (CVPR 2024 · 79 citations)
- 2025 · Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production (AAAI 2025 · 25 citations)
- 2026 · Mitigating Error Accumulation in Co-Speech Motion Generation via Global Rotation Diffusion and Multi-Level Constraints (AAAI 2026 · 4 citations)
Earliest papers here
- Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions (AAAI 2020 · 48 citations)
- You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection (NeurIPS 2022 · 6 citations)
- Executing Your Commands via Motion Diffusion in Latent Space (CVPR 2023 · 341 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
HumanML3D · FID (lower is better)
7 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 0.473 | Executing Your Commands via Motion Diffusion in Latent Space (CVPR 2023 · 341 citations) |
| 2024 | 0.084 | Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · 7 citations) |
| 2025 | 0.032 | LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning (ICLR 2025 · 0 citations) |
HumanML3D test · FID (lower is better)
7 papers, 2023–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 0.116 | Generating Human Motion From Textual Descriptions With Discrete Representations (CVPR 2023 · 311 citations) |
HumanML3D · R-Precision Top-1 (higher is better)
5 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 0.513 | FLAME: Free-Form Language-Based Motion Synthesis & Editing (AAAI 2023 · 139 citations) |
| 2025 | 0.557 | LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning (ICLR 2025 · 0 citations) |
KIT-ML · FID (lower is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.367 | Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · 7 citations) |
| 2025 | 0.14 | StickMotion: Generating 3D Human Motions by Drawing a Stickman (CVPR 2025 · 4 citations) |
AIST++ · FIDk (lower is better)
4 papers, 2022–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 25.96 | You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection (NeurIPS 2022 · 6 citations) |
| 2023 | 17.25 | UDE: A Unified Driving Engine for Human Motion Generation (CVPR 2023 · 55 citations) |
| 2025 | 15.7 | Controllable 3D Dance Generation Using Diffusion-Based Transformer U-Net (AAAI 2025 · 3 citations) |
HumanML3D test · R Precision (top 3) (higher is better)
4 papers, 2023–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 0.785 | Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs (NeurIPS 2023) |
What has been tried
73 papers here report 802 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 4%, hurt 27%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add human attribute pre-training | TikTok test · FID | 61.06 -> 38.19 | Helped | DisCo: Disentangled Control for Realistic Human Dance Generation (CVPR 2024 · 79 citations) |
| use training masking ratio Uniform .3-1 instead of Uniform 0-1 | HumanML3D · FID | 0.097 -> 0.089 | Helped | MMM: Generative Masked Motion Model (CVPR 2024 · 63 citations) |
| Add characteristic dance primitives, mirroring, and beat alignment | FineDance test set · FIDk | 60.91 -> 45.56 | Helped | Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives (CVPR 2024 · 56 citations) |
| use K = 16 frequency bands rather than K = 4 | Test set · FVD | 60.3 -> 47.1 | Helped | Generative Image Dynamics (CVPR 2024 · 48 citations) |
| add the anatomically informed GNN pose encoder | How2Sign test · MPVPE | 37.51 -> 31.47 | Helped | Neural Sign Actors: A Diffusion Model for 3D Sign Language Production from Text (CVPR 2024 · 44 citations) |
| remove gradient normalization | noisy motion refinement, std. 1 cm, HumanML3D · MPJPE all (cm) | 8.7 -> 30.2 | Hurt | Optimizing Diffusion Noise Can Serve As Universal Motion Priors (CVPR 2024 · 34 citations) |
| Use blended positional encodings instead of absolute-only encodings | Babel subsequences · R-prec | 0.699±0.004 -> 0.702±0.004 | Helped | Seamless Human Motion Composition with Blended Positional Encodings (CVPR 2024 · 32 citations) |
| Add the second take to refine transitions | HumanML3D test · FID | 1.00 -> 0.60 | Helped | Human Motion Diffusion as a Generative Prior (ICLR 2024 · 32 citations) |
| Add ID to the diffusion baseline | PHOENIX14T TEST · BLEU-1 | 21.50 -> 23.46 | Helped | Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production (AAAI 2025 · 25 citations) |
| Replace concatenation of actor and reactor features with addition | NTU120-AS, test-conditioned · FID | 10.999±0.7425 -> 12.828±0.9335 | Hurt | ReGenNet: Towards Human Action-Reaction Synthesis (CVPR 2024 · 25 citations) |
| add motion representation reformation | HumanML3D · FID | 2.196 -> 0.116 | Helped | Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · 24 citations) |
| Add beat conditioning | AIST++ · BA | 0.2235 -> 0.2366 | Helped | Bidirectional Autoregessive Diffusion Model for Dance Generation (CVPR 2024 · 16 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| MoMask | HumanML3D | FID | 0.045 – 0.116 | 2 papers, e.g. ICLR 2025 and CVPR 2025 |
| MoMask | KIT-ML | FID | 0.204 – 0.411 | 2 papers, e.g. ICLR 2025 and CVPR 2025 |
| FACT* | AIST++ | FIDk | 35.35 – 53.98 | 2 papers, e.g. NeurIPS 2022 and AAAI 2025 |
| MDM | HumanML3D | FID | 0.423 – 0.544 | 2 papers, e.g. ICLR 2025 and CVPR 2023 |
| MDM | HumanML3D test | FID | 0.544 – 0.698 | 2 papers, e.g. AAAI 2024 and ICLR 2024 |
| MoMask | KIT-ML | R-Precision Top 1 | 0.369 – 0.433 | 2 papers, e.g. CVPR 2025 and ICLR 2025 |
What reviewers will expect
Datasets and baselines named in the 39 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HumanML3D · 12 of 39 papers
- KIT-ML · 5 of 39 papers
- AMASS · 4 of 39 papers
- BEHAVE · 2 of 39 papers
- HumanAct12 · 2 of 39 papers
- InterHuman · 2 of 39 papers
- BABEL · 2 of 39 papers
Baselines
- MDM · 10 of 39 papers
- T2M-GPT · 7 of 39 papers
- MotionDiffuse · 6 of 39 papers
- MoMask · 5 of 39 papers
- MLD · 5 of 39 papers
- T2M · 4 of 39 papers
- EDGE · 3 of 39 papers
- ReMoDiffuse · 3 of 39 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 138 papers; from OpenAlex and the papers' affiliations).
- Tencent · 7 papers
- Zhejiang University · 6 papers
- Nanyang Technological University · 5 papers
- Tsinghua University · 5 papers
- Nanjing University of Science and Technology · 4 papers
- Tel Aviv University · 4 papers
- Tianjin AiForward Science and Technology Co · 3 papers
- Shanghai Jiao Tong University · 3 papers
- The Chinese University of Hong Kong · 3 papers
- Korea University · 2 papers
Use it
Download BibTeX of 30 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Diffusion-based human image animation (126 papers)
- Diffusion-based video editing (92 papers)
- Talking face generation (81 papers)
- Video-to-audio synthesis (76 papers)
Elsewhere in the topic
- Efficient video diffusion models (283 papers)
- Trajectory-guided motion control (184 papers)
- Compositional text-to-video diffusion (159 papers)
- Autoregressive long video generation (156 papers)
- World-model video generation (110 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).