Research map / Video / Video generation
Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spatiotemporal Skip Guidance (STG), a training-free guidance method that builds an implicit weak model by skipping spatial/temporal transformer layers in video diffusion models, improving quality without hurting diversity or motion.
Problem
Guidance methods like CFG reduce diversity and motion in video diffusion, while Autoguidance needs training an extra weak model, which is impractical for large video models; existing self-perturbation methods (PAG, SEG) target 2D image attention only.
Approach
Create an aligned implicit weak model by skipping spatiotemporal layers (residual or attention skip, applied separately to spatial and temporal layers for factorized attention) of the same network, and use the difference as guidance, extending PAG-style self-perturbation to video.
Key results
- On SVD (UCF-101 I2V), STG reduces FVD from 151.3 (CFG) to 128.7 and raises Dynamic Degree from 0.562 to 0.694.
- On Mochi (VBench T2V), STG raises Imaging Quality from 0.524 to 0.628 and Aesthetic Quality from 0.507 to 0.554 over CFG.
- Ablation on SVD shows spatial guidance and temporal guidance each improve FVD (151.3 to 133.8 to 128.7).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add STG to CFG | Mochi · VBench T2V | Imaging Quality: 0.524 -> 0.628 | Helped |
| Add STG to CFG | Mochi · VBench T2V | Dynamic Degree: 0.87 -> 0.86 | Hurt |
| Add STG to CFG | Open-Sora · VBench T2V | Imaging Quality: 0.561 -> 0.606 | Helped |
| Add STG to CFG | Open-Sora · VBench T2V | Dynamic Degree: 0.902 -> 0.895 | Hurt |
| Add STG to CFG | SVD · UCF-101 I2V | FVD: 151.3 -> 128.7 | Helped |
| Add STG to CFG | SVD · VBench I2V | Dynamic Degree: 0.562 -> 0.694 | Helped |
| Add spatial guidance to CFG (ablation) | SVD · UCF-101 I2V | FVD: 151.3 -> 133.8 | Helped |
| Add temporal guidance to CFG (ablation) | SVD · UCF-101 I2V | FVD: 151.3 -> 128.7 | Helped |
| Use attention skip instead of residual skip (ablation) | SVD STG · UCF-101 I2V | FVD: 155.9 -> 128.7 | Helped |
| Use residual skip instead of attention skip (ablation) | Mochi STG · VBench T2V | Imaging Quality: 0.555 -> 0.628 | Helped |
| Use attention skip instead of residual skip (ablation) | Open-Sora STG · VBench T2V | Imaging Quality: 0.550 -> 0.606 | Helped |
Surprising findings
- STG-R (residual skip) works better on Mochi while attention skip works better on Open-Sora and SVD, attributed to model depth and factorized attention.
Limitations
- Optimal scale and layer selection vary across models and require heuristic tuning.
- Dynamic Degree and motion metrics show marginal gains on some benchmarks near saturation.
- Evaluation uses subsets of prompts (e.g., 100 for Mochi) due to computational limits.
Experimental setup
Datasets: UCF-101 VBench prompts EvalCrafter prompts
Benchmarks: UCF-101 VBench EvalCrafter
Metrics: FVD Inception Score Imaging Quality Aesthetic Quality Motion Smoothness Dynamic Degree Temporal Flickering
Baselines: CFG Autoguidance PAG SEG
Builds on: Classifier-Free Guidance Autoguidance Perturbed Attention Guidance Smoothed Energy Guidance Rescaling classifier-free guidance Restart sampling
Closest papers by meaning
- Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs (CVPR 2025 · similarity 0.90)
- History-Guided Video Diffusion (ICML 2025 · similarity 0.89)
- ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler (ICLR 2025 · similarity 0.88)
- Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.88)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.88)
- SkipSR: Faster Super-Resolution with Token Skipping (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.