Research map / Image generation / Fast diffusion sampling

Revisiting Diffusion Model Predictions Through Dimensionality

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a generalized prediction target u = k x - (1-k) n with a linear-diffusion-model theory giving k* = D/(D+d), and k-Diff, which learns k end-to-end as a single scalar, evaluated on ImageNet 256 and 512 in latent and pixel space.

Problem

Diffusion and flow matching models have shifted from epsilon/v to x prediction in high dimensions, but there is no formal explanation of how the optimal prediction target depends on data dimensionality, and no principled way to choose it without expensive search.

Approach

Generalizes the prediction target to a linear combination k x - (1-k) n; analyzes a single linear layer under a manifold-projected whitened data assumption to get k* = D/(D+d); and makes k a learnable sigmoid-parameterized scalar trained by backpropagation.

Key results

• Latent space (LightningDiT-XL/1, ImageNet 256, 800 epochs): k-Diff reaches FID 1.34 vs 1.35 for LightningDiT-XL/1 as reported, with learned k converging to 0.66.
• Pixel space (JiT-B/16, ImageNet 256): k-Diff reaches FID 3.64 with IS 274.9 vs 3.66 for JiT-B/16, with k converging near 1.0 (x-prediction).
• ImageNet 512 pixel (JiT-B/32): k-Diff FID 4.03 vs 4.02 for JiT-B/32, so it does not improve on the baseline there.

Surprising findings

• In latent space, the learned k=0.66 lies between v- and x-prediction, and time-dependent k(t) degraded FID to 2.17 versus 2.05 for constant k.
• k-Diff's reported FID at 512 pixel-space (4.03) is slightly worse than the JiT-B/32 baseline (4.02).

Limitations

• Theory relies on a single linear layer, whitened data, and orthogonal projection onto an intrinsic subspace, which are strong simplifications for real image manifolds.
• Gains over fixed-target baselines are marginal (e.g., 1.34 vs 1.35 FID, 3.64 vs 3.66), and comparisons use different training/sampling settings than originally reported.
• Only ImageNet class-conditional generation is evaluated, with a single seed implied and no variance reported.

Experimental setup

Datasets: ImageNet

Benchmarks: ImageNet 256x256 ImageNet 512x512

Metrics: FID IS

Baselines: DiT-XL/2 SiT-XL/2 REPA DDT-XL/2 RAE ADM-G RIN SiD VDM++ PixelFlow PixNerd JiT-B/16

Builds on: flow matching LightningDiT JiT EDM REPA

Closest papers by meaning

• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.90)
• Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifold (NeurIPS 2026 · similarity 0.89)
• Elucidating Flow Matching ODE Dynamics via Data Geometry and Denoisers (ICML 2025 · similarity 0.89)
• Diffusion Models Encode the Intrinsic Dimension of Data Manifolds (ICML 2024 · similarity 0.89)
• Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch (NeurIPS 2025 · similarity 0.89)
• Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.88)