Research map / Image generation / Fast diffusion sampling
Critical windows: non-asymptotic theory for feature emergence in diffusion models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Develops non-asymptotic theory bounding the time windows ("critical windows") during diffusion sampling when features of the output are determined, for mixtures of strongly log-concave components, and validates with synthetic Gaussian mixtures and a small Stable Diffusion 2.1 study.
Problem
Empirically, diffusion models decide certain image features (class, color, background, gender) within narrow time windows of sampling, but there is no rigorous theory explaining why this happens given the continuous-time nature of diffusion.
Approach
Formalizes critical windows via a noise-then-denoise "S-targeted reverse process" on mixtures; proves TV-closeness to a sub-mixture for times between a T_lower (sub-mixtures become indistinguishable under forward noising) and T_upper (target and outside components remain separated), with explicit log-distance formulas for Gaussian mixtures and hierarchical mixture trees.
Key results
- On a CIFAR-10 DDPM membership inference test, the proposed NoiseDenoise attack reaches AUC 0.6636 and TPR@0.01 FPR 0.023, below SecMI_stat (AUC 0.8847) and SecMI_nn (AUC 0.9132), though it flags some training points those methods miss.
- On SD2.1 car images, background agreement drops by about 25% between steps 480 and 490, and size drops about 15% from 470 to 490, with color changing later at 450-470.
- For SD2.1 laboratory technician portraits, gender agreement falls from over 80% to about 50% between t=80 and t=84.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NoiseDenoise instead of SecMI_stat | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | AUC: $.8847 -> $.6636 | Hurt |
| Use NoiseDenoise instead of SecMI_nn | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | AUC: $.9132 -> $.6636 | Hurt |
| Use NoiseDenoise instead of SecMI_stat | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | TPR at FPR .01: $.073 -> $.023 | Hurt |
| Use NoiseDenoise instead of SecMI_nn | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | TPR at FPR .01: $.245 -> $.023 | Hurt |
| Use NoiseDenoise instead of SecMI_stat | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | TPR at FPR .05: $.344 -> $.14 | Hurt |
| Use NoiseDenoise instead of SecMI_nn | DDPM membership inference on CIFAR-10 · CIFAR-10 membership inference | TPR at FPR .05: $.609 -> $.14 | Hurt |
| Increase noising and denoising time from 480 to 490 | SD2.1 car images · SD2.1 car images | background feature agreement: 25% drop | Hurt |
| Increase noising and denoising time from 470 to 490 | SD2.1 car images · SD2.1 car images | size feature agreement: 15% drop | Hurt |
Surprising findings
- The proposed NoiseDenoise membership inference attack underperforms the SecMI baselines in AUC and low-FPR TPR, though it identifies some training points they miss.
Limitations
- Upper-bound on critical window has a logarithmic dependence on dimension, and the theory requires strong statistical separation of components inside and outside the target.
- Theory assumes perfect score and continuous time, ignoring discretization and score error.
- Stable Diffusion experiments are preliminary, with feature labels produced by CLIP prompts on small samples.
Experimental setup
Datasets: CIFAR-10 synthetic Gaussian mixtures
Benchmarks: CIFAR-10 membership inference
Metrics: TV distance bounds AUC TPR at low FPR feature agreement percentage
Baselines: SecMI_stat SecMI_nn
Builds on: score-based generative modeling Girsanov theorem DDPM
Closest papers by meaning
- Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution (NeurIPS 2025 · similarity 0.89)
- Blink of an eye: a simple theory for feature localization in generative models (ICML 2025 · similarity 0.88)
- Feature Information Dynamics in Diffusion (NeurIPS 2026 · similarity 0.88)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.88)
- Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.88)
- On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.