Research map / Video / Video generation
Efficient video diffusion models
Builds video diffusion models with latent decomposition, autoregressive distillation and training-free long-video generation.
283 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2022–2026) · an approach in Video generation, keyword cluster “video generation · video diffusion · text video”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 2 of 3 tracked benchmarks.
45 papers in 2023–24 and 235 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 19.7% to 21.0%.
Open questions the data points to
- MSR-VTT (FVD): best went from 91.3 (up to 2024) to 64 in 2025 — Generative Pre-trained Autoregressive Diffusion Transformer (NeurIPS 2025 · 1 citation)
- UCF-101 (IS): best went from 62.88 (up to 2024) to 70.53 in 2025 — WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model (CVPR 2025 · 18 citations)
- Papers report FILM on Pexels (FVD) anywhere from 371.83 to 559.16: comparisons against it depend on whose number you take.
- Papers report DIGAN on Sky Time-lapse (FVD) anywhere from 83.1 to 114.6: comparisons against it depend on whose number you take.
What looks settled
- UCF-101 (FVD): best up to 2024 180.9, best in 2025–26 214 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2022 · Video Diffusion Models (NeurIPS 2022 · 204 citations)
- 2023 · Conditional Image-to-Video Generation With Latent Flow Diffusion Models (CVPR 2023 · 117 citations)
- 2024 · Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution (CVPR 2024 · 78 citations)
- 2025 · From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · 37 citations)
- 2026 · Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers (AAAI 2026 · 3 citations)
Earliest papers here
- Video Diffusion Models (NeurIPS 2022 · 204 citations)
- Flexible Diffusion Modeling of Long Videos (NeurIPS 2022 · 102 citations)
- MCVD - Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation (NeurIPS 2022 · 45 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
UCF-101 · FVD (lower is better)
6 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 180.9 | SF-V: Single Forward Video Generation Model (NeurIPS 2024 · 1 citation) |
MSR-VTT · FVD (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 91.3 | TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models (CVPR 2024 · 21 citations) |
| 2025 | 64 | Generative Pre-trained Autoregressive Diffusion Transformer (NeurIPS 2025 · 1 citation) |
UCF-101 · IS (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 62.88 | Grid Diffusion Models for Text-to-Video Generation (CVPR 2024 · 17 citations) |
| 2025 | 70.53 | WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model (CVPR 2025 · 18 citations) |
What has been tried
123 papers here report 1,314 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 4%, hurt 21%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add the finetuned VAE-Decoder | YouHQ40 · PSNR | 23.82 -> 25.47 | Helped | Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution (CVPR 2024 · 78 citations) |
| Add neighbor-frame features during decoding and use MaxCA blocks | Middlebury · LPIPS | 0.077 -> 0.019 | Helped | LDMVFI: Video Frame Interpolation with Latent Diffusion Models (AAAI 2024 · 76 citations) |
| add explicit conditioning on the original high-resolution start and end frames during training | video interpolation · FID | Helped | Video Interpolation with Diffusion Models (CVPR 2024 · 42 citations) | |
| Use asymmetric DMD with a bidirectional teacher instead of a causal teacher | 10-second text-to-video · temporal quality | 91.9 -> 94.7 | Helped | From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · 37 citations) |
| Add 360 Enhancement Techniques (latent rotation, circular padding, latitude-aware loss) to AnimateDiff+LoRA | User study (31 participants) · user preference % across criteria | Graphics 14.1→23.0, End Continuity 6.0→16.9, Content 12.1→16.1, Motion 6.5→14.5 | Helped | 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model (CVPR 2024 · 34 citations) |
| add the Dysen module | UCF-101 · FVD | 346.40 -> 255.42 | Helped | Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs (CVPR 2024 · 34 citations) |
| Fine-tune temporal attention layers rather than self- and cross-attention layers | Motion customization · Frame-transition smoothness | Helped | VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models (CVPR 2024 · 34 citations) | |
| Replace DMD with SiD as the distribution-matching objective | VBench · Quality Score | 85.07 -> 85.52 | Helped | Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025 · 33 citations) |
| add single-frame depth datasets to multi-frame training | ScanNet++ · AbsRel | 0.201 -> 0.176 | Helped | Learning Temporally Consistent Video Depth from Video Diffusion Priors (CVPR 2025 · 28 citations) |
| Use the motion-content decoupled pipeline | A red horse runs in the sky · visual quality and content generalization | Helped | LAMP: Learn A Motion Pattern for Few-Shot Video Generation (CVPR 2024 · 25 citations) | |
| Add motion magnitude reconstruction loss | Image-to-4D · CLIP-O | 0.73 -> 0.75 | Helped | Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (NeurIPS 2024 · 24 citations) |
| add video features to unconditional generation | BGM909 test set · PCHE | 3.189 -> 2.835 | Helped | Diff-BGM: A Diffusion Model for Video Background Music Generation (CVPR 2024 · 21 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| FILM | Pexels | FVD | 371.83 – 559.16 | 2 papers, e.g. ICLR 2025 and ICLR 2025 |
| DIGAN | Sky Time-lapse | FVD | 83.1 – 114.6 | 2 papers, e.g. CVPR 2025 and CVPR 2023 |
What reviewers will expect
Datasets and baselines named in the 113 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- VBench prompts · 17 of 113 papers
- DAVIS · 10 of 113 papers
- UCF-101 · 9 of 113 papers
- RealEstate10K · 8 of 113 papers
- Panda70M · 7 of 113 papers
- SkyTimelapse · 5 of 113 papers
- MovieGenBench · 5 of 113 papers
- VBench · 5 of 113 papers
Baselines
- PAB · 8 of 113 papers
- FreeNoise · 6 of 113 papers
- MotionCtrl · 6 of 113 papers
- CogVideoX-5B · 6 of 113 papers
- SVG · 6 of 113 papers
- SVD · 5 of 113 papers
- FreeInit · 5 of 113 papers
- T-GATE · 5 of 113 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 280 papers; from OpenAlex and the papers' affiliations).
- Nanyang Technological University · 13 papers
- Korea Advanced Institute of Science and Technology · 12 papers
- Google · 11 papers
- Tsinghua University · 9 papers
- Chinese Academy of Sciences · 8 papers
- Alibaba Group · 8 papers
- Zhejiang University · 8 papers
- Hong Kong University of Science and Technology · 7 papers
- Adobe · 6 papers
- Fudan University · 6 papers
Use it
Download BibTeX of 26 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Compositional text-to-video diffusion (159 papers)
- Trajectory-guided motion control (184 papers)
- Autoregressive long video generation (156 papers)
- World-model video generation (110 papers)
Elsewhere in the topic
- Text-to-motion diffusion (140 papers)
- Diffusion-based human image animation (126 papers)
- Diffusion-based video editing (92 papers)
- Talking face generation (81 papers)
- Video-to-audio synthesis (76 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).