Research map / Image generation / Fast diffusion sampling
Infinite-Resolution Integral Noise Warping for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes infinite-resolution integral noise warping, which samples Brownian bridge increments instead of upsampling noise, giving grid and particle variants that preserve Gaussian white noise for temporally consistent video diffusion.
Problem
Training-free noise warping for video generation with image diffusion models must preserve the Gaussian white noise distribution while adding temporal correlation, but the existing integral noise warping method (HIWYN) is computationally expensive due to upsampling.
Approach
Taking the upsampling resolution to infinity yields a Brownian bridge scaling limit; warped noise pixels are formed by autoregressively sampling Brownian bridge increments over overlap areas, removing upsampling entirely. A particle-based bilinear-kernel partition replaces polygon clipping, avoiding degenerate overlaps and extending to 3D.
Key results
- Grid-based variant matches HIWYN at infinite resolution while running 8.0x (GPU) to 19.7x (CPU) faster and using 9.22x less memory.
- Particle-based variant gives a further 5.21x speed-up over the grid variant (41.7x over HIWYN on GPU) and warps 1024x1024 noise in ~0.0086s.
- Both variants produce noise indistinguishable from white Gaussian noise under K-S and Moran's I tests, unlike generic warping baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the grid-based infinite-resolution warp instead of HIWYN at N=8 | Noise warping · 1024×1024 noise warp, GPU | speedup: 8.0× faster | Helped |
| use the grid-based infinite-resolution warp instead of HIWYN at N=8 | Noise warping · 1024×1024 noise warp, CPU | speedup: 19.7× faster | Helped |
| use the particle-based warp instead of HIWYN at N=8 | Noise warping · 1024×1024 noise warp, GPU | speedup: 41.7× speedup | Helped |
| use grid-based infinite-resolution warping instead of HIWYN at N=8 | Noise warping · 1024×1024 noise warp | memory reduction: 9.22× less memory | Helped |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Church | Consistency: 9.868e-2 -> 1.065e-1 | Mixed |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Cat | Consistency: 6.001e-2 -> 5.898e-2 | Helped |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Church | Realism: 4.643e-2 -> 5.180e-2 | Hurt |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Cat | Realism: 1.559e-1 -> 1.496e-1 | Helped |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Church | Faithfulness: 3.872e-2 -> 4.309e-2 | Hurt |
| use particle-based rather than grid-based warping | Infinite-resolution noise warp · Cat | Faithfulness: 2.039e-2 -> 2.064e-2 | Hurt |
Surprising findings
- Particle-based variant shows no visible quality loss against the grid-based variant in real-world video generation despite being only approximate.
Limitations
- Particle-based variant does not capture temporal correlations from contraction or expansion.
- Connection between initial noise consistency and generated output consistency is empirical only.
- Not evaluated for latent diffusion models.
Experimental setup
Datasets: real-world videos (church, cat scenes)
Benchmarks: Moran's I Kolmogorov-Smirnov test warp MSE consistency on church and cat scenes
Metrics: Moran's I K-S test p-value 2-Wasserstein distance runtime memory usage IoU Chamfer distance consistency (warp MSE) realism faithfulness
Baselines: HIWYN PYoCo Control-A-Video Random noise Fixed noise Bilinear interpolation Nearest interpolation
Builds on: Integral noise warping (HIWYN) SDEdit Perturbed-Attention Guidance RAFT
Closest papers by meaning
- How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models (ICLR 2024 · similarity 0.91)
- Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment (NeurIPS 2024 · similarity 0.88)
- Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.87)
- $\infty$-Diff: Infinite Resolution Diffusion with Subsampled Mollified States (ICLR 2024 · similarity 0.87)
- FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise (ICLR 2025 · similarity 0.87)
- Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.