Research map / Video / Video generation
IV-mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IV-mixed Sampler, a training-free sampling scheme that alternates image diffusion model (SD V1.5) and video diffusion model denoising steps via DDIM-inversion and DDIM to improve text-to-video quality.
Problem
Video diffusion models produce low-quality videos with weak semantic faithfulness, largely because they are trained on small, low-quality video datasets; existing inference-time scaling gives only minimal visible gains.
Approach
Inject image diffusion model (IDM) guidance into each video diffusion sampling step by DDIM-inversion followed by DDIM denoising with the IDM and VDM alternately (the "IV-IV" combination), framed as an ODE with separate CFG scales for IDM and VDM.
Key results
- On Chronomagic-Bench-150 with Animatediff, UMT-FVD drops from 275.18 (standard DDIM) to 228.59 with IV-mixed Sampler.
- On Chronomagic-Bench-1649, IV-mixed Sampler achieves the best UMT-FVD, UMTScore, and GPT4o-MTScore across VideoCrafterV2, ModelScope-T2V, and Animatediff, e.g., 219.29 to 192.72 UMT-FVD on Animatediff.
- On MSR-VTT with ModelScope-T2V, FVD (StyleGAN) falls from 739.12 to 603.31.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-150 | UMT-FVD: 275.18 -> 228.59 | Helped |
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-150 | UMTScore: 2.82 -> 3.30 | Helped |
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-150 | GPT4o-MTScore: 2.83 -> 3.55 | Helped |
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-1649 | UMT-FVD: 219.29 -> 192.72 | Helped |
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-1649 | UMTScore: 3.08 -> 3.40 | Helped |
| Apply IV-mixed Sampler (IV-IV) instead of standard sampling | Animatediff (SD V1.5, Motion Adapter V3) · Chronomagic-Bench-1649 | GPT4o-MTScore: 2.62 -> 3.34 | Helped |
| Apply IV-mixed Sampler instead of standard sampling | Animatediff · UCF-101-FVD | FVD (StyleGAN): 815.08 -> 800.45 | Helped |
| Apply IV-mixed Sampler instead of standard sampling | Animatediff · UCF-101-FVD | FVD (VideoGPT): 819.93 -> 804.88 | Helped |
| Apply IV-mixed Sampler instead of standard sampling | Animatediff · MSR-VTT-FVD | FVD (StyleGAN): 762.23 -> 721.32 | Helped |
| Apply IV-mixed Sampler instead of standard sampling | Animatediff · MSR-VTT-FVD | FVD (VideoGPT): 761.01 -> 719.9 | Helped |
| Use SD V1.5 instead of Mini SD as the IDM (ablation) | IV-mixed Sampler on ModelScope-T2V · Chronomagic-Bench-150 | UMT-FVD: 247.99 -> 234.90 | Helped |
| Apply VV-VV over 50.0% rather than standard DDIM sampling (ablation) | VideoCrafterV2 · Chronomagic-Bench-150 | UMT-FVD: 214.06 -> 208.67 | Helped |
Surprising findings
- Adding Gaussian noise in the R-[.] variant (SDEdit-like) substantially degrades video quality compared to standard DDIM, whereas deterministic DDIM-inversion variants improve it.
- Applying IV-IV at every step on VideoCrafterV2 destroys temporal coherence, so the paper switches to VV-VV for the first 66.7% of steps.
Limitations
- Substantially higher inference cost (about 4.4x runtime) than standard sampling.
- Performance ceiling depends on finding a compatible IDM sharing the VDM's latent space; ModelScope-T2V lacked a suitable high-resolution IDM.
- On Animatediff and ModelScope-T2V it does not beat all baselines on UMT-FVD in Chronomagic-Bench-150 (I4VGen is better on some metrics).
Experimental setup
Datasets: UCF-101 MSR-VTT Chronomagic-Bench-150 Chronomagic-Bench-1649
Benchmarks: UCF-101-FVD MSR-VTT-FVD Chronomagic-Bench-150 Chronomagic-Bench-1649
Metrics: UMT-FVD UMTScore GPT4o-MTScore FVD (StyleGAN) FVD (VideoGPT)
Baselines: Standard DDIM FreeInit I4VGen Unictrl Standard DPM-Solver++
Builds on: DDIM SDEdit FreeInit I4VGen classifier-free guidance
Closest papers by meaning
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.90)
- Towards Precise Scaling Laws for Video Diffusion Transformers (CVPR 2025 · similarity 0.90)
- ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler (ICLR 2025 · similarity 0.90)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.89)
- VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guide (CVPR 2025 · similarity 0.89)
- Hierarchical Patch Diffusion Models for High-Resolution Video Generation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.