Research map / Video / Video generation
Spectral Progressive Diffusion for Efficient Image and Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spectral Progressive Diffusion, which grows resolution along the denoising trajectory of pretrained image and video diffusion models using spectral noise expansion and a power-spectrum-derived transition schedule, with training-free and LoRA fine-tuning variants.
Problem
Diffusion transformers scale quadratically with token count, while high-resolution computation in early denoising steps is largely redundant because low frequencies emerge first; existing progressive-resolution methods either need architectural changes/training from scratch or rely on heuristic tran
Approach
Expand resolution at timesteps when high frequencies emerge from noise, using a DCT-based spectral noise expansion that fills new high-frequency slots with scaled noise plus a timestep re-alignment, and derive transition times analytically from a power-law power spectrum with a single tolerance delta.
Key results
- On FLUX.1-dev at 7-step tier, Ours (S=3) reaches 7.09x wall-clock speedup and 406.24 TFLOPs with NIQE 5.99, versus RALU 6.69x and NIQE 6.87.
- On Z-Image, LoRA fine-tuning (S=2) raises ImageReward from 0.904 (training-free) to 0.982 at 1.65x speedup, close to the 50-step baseline of 0.965.
- On WAN 2.1 video, training-free S=2 gives 2.03x speedup with VBench Image Quality 0.6114 versus 0.6133 for the 50-step baseline and 0.5706 for the 25-step baseline.
Surprising findings
- The training-free S=2 Z-Image result underperforms the baseline on ImageReward (0.904 vs 0.965), but LoRA fine-tuning recovers and exceeds it (0.982).
Limitations
- Assumes the pretrained model generalizes to intermediate resolutions not seen in training for training-free use.
- Derivation uses a zero-mean Gaussian per-frequency modelling assumption and only spatial spectra; temporal frequencies not addressed.
- Fine-tuning experiments limited to 2000 iterations of LoRA rank 32 and only a few models.
Experimental setup
Datasets: Aesthetics-Train-V2 VChitect-T2V-Dataverse MS-COCO validation prompts
Benchmarks: ImageReward CLIP-IQA NIQE T2I-CompBench GenEval VBench
Metrics: ImageReward CLIP-IQA NIQE T2I-CompBench GenEval VBench dimensions TFLOPs wall-clock speedup
Baselines: FLUX.1-dev 50/10/7 steps RALU Bottleneck Sampling WAN 2.1 25 steps Z-Image 50 steps PixelGen 25 steps
Builds on: flow matching RALU Bottleneck Sampling LoRA spectral autoregression
Closest papers by meaning
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.91)
- Boosting Diffusion Models with Moving Average Sampling in Frequency Domain (CVPR 2024 · similarity 0.90)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.90)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise (ICLR 2025 · similarity 0.90)
- FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.