Research map / Image generation / Fast diffusion sampling

LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Training-free diffusion acceleration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LENS, a lightweight transformer that modulates low-frequency PCA coefficients of diffusion initial noise to improve distilled text-to-image models' quality at low inference overhead.

Problem

Distilled diffusion models generate fast but with degraded quality; existing noise-optimization fixes are either costly test-time optimization (ReNo) or hypernetworks (HyperNoise) that reuse the full diffusion backbone and operate in full high-dimensional latent noise space.

Approach

Restrict noise modulation to low-frequency patch-wise PCA coefficients of the latent noise (justified by an assumption that high-frequency components have bounded reward effect and a KL bound), and train a standalone transformer on those coefficients with a reward-regularized objective, decoupled from the diffusion backbone.

Key results

• On SD-Turbo, LENS reaches GenEval2 mean 35.69 vs 34.96 for HyperNoise and 33.95 for the unmodified model, at 0.112 s vs 0.194 s.
• LENS reduces noise-modulation FLOPs from 997.1 G to 2.4 G on SD-Turbo and parameters from 136.5 M to 4.8 M, with inference-time 5.68 ms vs 70.78 ms.
• On SANA-Sprint, LENS mean 45.79 is comparable to HyperNoise 45.92 while being faster (0.352 s vs 0.532 s); ReNo reaches 45.25 at 14.964 s.

Surprising findings

• On SANA-Sprint, LENS is slightly below HyperNoise on mean GenEval2 (45.79 vs 45.92), and on Hyper-SDXL HyperNoise is higher on position and verb.
• LENS lowers LPIPS diversity relative to random noise (0.5887 vs 0.6249), though it keeps higher LPIPS than HyperNoise.

Limitations

• Relies on a fixed PCA basis from ImageNetV2 that may not transfer to domains far from the reference data.
• Low-frequency-only modulation may limit refinement of fine details governed by high-frequency components.
• Key assumption (bounded high-frequency reward contribution, epsilon) is empirically checked only, and the Jacobian bound M<1 is a strong assumption for the approximation.

Experimental setup

Datasets: ImageNetV2 ABC-6K T2I-CompBench Pick-a-Pic v2

Benchmarks: GenEval2

Metrics: GenEval2 mean and sub-scores (object, attribute, count, posi inference time FLOPs parameter count LPIPS DINOv2 score

Baselines: ReNo HyperNoise random noise initialization

Builds on: HyperNoise ReNo LoRA Patch PCA analysis of diffusion noise

Closest papers by meaning

• FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.89)
• Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.88)
• One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls (CVPR 2024 · similarity 0.88)
• Boosting Diffusion Models with Moving Average Sampling in Frequency Domain (CVPR 2024 · similarity 0.88)
• Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models (ICLR 2025 · similarity 0.88)
• Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network (NeurIPS 2026 · similarity 0.87)