Research map / Video / Video generation
History-Guided Video Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Forcing Transformer (DFoT), a video diffusion model with per-frame independent noise levels that enables conditioning on arbitrary history, and History Guidance methods (vanilla, temporal, fractional, and combined) that improve video quality, consistency, and dynamics, including long video rollouts.
Problem
Classifier-free guidance with variable-length video history is incompatible with standard video diffusion architectures (fixed-size conditioning), and binary-dropout history training performs poorly due to inefficient token utilization.
Approach
Treat history frames as noise-free frames within a single non-causal transformer trained with independent per-frame noise levels (noise-as-masking), so any subset of history can be used as a conditioning signal; then compose conditional scores over different history lengths (temporal) and noise levels (fractional/frequency) for guidance.
Key results
- On Kinetics-600, DFoT trained from scratch achieves FVD 4.3, outperforming standard diffusion (SD, FVD 4.8) and binary-dropout diffusion (BD, FVD 6.4) under the same architecture.
- Fractional history guidance lowers the best FVD of vanilla history guidance from 181.6 to 170.4 on 64-frame Kinetics-600 rollouts, versus SD at 247.5 and DFoT without guidance at 208.0.
- Temporal history guidance improves Minecraft long-context FVD from 97.63 to 79.19 and achieves 83% success on the robotic Fruit Swapping task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with independent per-frame noise levels instead of framewise binary dropout (ablation) | DFoT · Kinetics-600 | FVD: 6.4 ± 0.1 -> 4.3 ± 0.1 | Helped |
| Apply vanilla history guidance | DFoT · Kinetics-600, 64-frame rollout | frame quality and consistency | Helped |
| Increase vanilla history guidance scale beyond ω = 1.5 | DFoT with vanilla history guidance · Kinetics-600, 64-frame rollout | FVD | Hurt |
| Guide with fractionally masked history rather than clean history alone | DFoT · Kinetics-600, 64-frame rollout | FVD: 181.6 -> 170.4 | Helped |
| Guide with lower frequencies by increasing history noise | DFoT with fractional history guidance · Kinetics-600, 64-frame rollout | video dynamics | Helped |
| Compose scores from long- and short-context histories | DFoT with temporal history guidance · Minecraft | FVD: 97.63 -> 79.19 | Helped |
| Compose full-history and single-frame scores | DFoT with temporal history guidance · Fruit Swapping | success rate: 83% | Helped |
Surprising findings
- Framewise binary-dropout history guidance performs poorly, with a significant FVD drop versus standard diffusion, attributed to inefficient token utilization.
- Vanilla history guidance at high guidance scales (w>=3) tends to produce static videos, which fractional guidance mitigates.
Limitations
- Fractional history guidance uses simple implementations; sophisticated frequency-aware sampling strategies are left to future work.
- Path/factorization consistency across multiple noising paths in the ELBO remains an open issue per the remark in the appendix.
- Vanilla history guidance at high guidance scales causes loss of diversity and static videos.
Experimental setup
Datasets: Kinetics-600 RealEstate10K Minecraft Fruit Swapping robot demonstrations
Benchmarks: Kinetics-600 RealEstate10K Minecraft VBench
Metrics: FVD VBench LPIPS success rate
Baselines: Standard Diffusion (SD) Binary-Dropout Diffusion (BD) Full-Sequence Diffusion with Reconstruction Guidance (FS) MAGVIT-v2 W.A.L.T Rolling Diffusion Video Diffusion MAGVIT
Builds on: Diffusion Forcing classifier-free guidance DiT Reconstruction guidance
Closest papers by meaning
- Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling (CVPR 2025 · similarity 0.89)
- FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling (ICLR 2024 · similarity 0.88)
- Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.88)
- Elucidating the design space of classifier-guided diffusion generation (ICLR 2024 · similarity 0.88)
- Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.