Research map / Image generation / Fast diffusion sampling
A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FLIPD, a single-diffusion-model local intrinsic dimension estimator derived from the Fokker-Planck equation, which avoids ODE solves and scales to Stable Diffusion latents.
Problem
Estimating the local intrinsic dimension (LID) of individual data points is expensive for traditional estimators and existing generative-model-based estimators are inaccurate, need multiple models, or do not scale to high-resolution images.
Approach
Shows that LIDL's regression slope (log density of Gaussian-convolved data vs noise level) can be computed from a single pre-trained diffusion model via a Fokker-Planck-derived closed-form expression involving the score's Jacobian trace and squared norm, giving an ODE-free estimator FLIPD(x,t0) = D + sigma^2(t0)[tr(grad s) + ||s||^2], with Hutchinson trace estimation and knee detection for t0.
Key results
- On the synthetic benchmark, FLIPD achieves the lowest MAE among model-based estimators in most settings (e.g., 12.8 vs 100 for NB and 24.9 for LIDL on U900 in R^1000) and perfect concordance index in all scenarios.
- FLIPD has Spearman correlation with PNG size of 0.837 on MNIST, 0.883 on FMNIST, 0.819 on CIFAR10, and 0.876 on SVHN, versus 0.480 for NB on FMNIST.
- FLIPD takes 0.10 s per MNIST/FMNIST image and 0.20 s per LAION image on Stable Diffusion latents, while NB exceeds 9x10^3 s for LAION images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FLIPD instead of the strongest model-based baseline | Synthetic LID estimation · Complex synthetic manifolds in the benchmark | MAE | Helped |
| Use FLIPD for pointwise LID ranking | Synthetic LID estimation · Scenarios with variable ground-truth LID | concordance index: 1.00 | Helped |
| Use FLIPD instead of NB | UNet-based image LID estimation · FMNIST and SVHN | Spearman correlation with PNG compression size: 0.480 -> 0.883; 0.573 -> 0.876 | Helped |
| Use FLIPD instead of NB | UNet-based image LID estimation · MNIST and CIFAR10 | Spearman correlation with PNG compression size: 0.864 vs 0.837; 0.894 vs 0.819 | Hurt |
| Use FLIPD instead of NB | UNet-based image LID estimation · MNIST/FMNIST and SVHN/CIFAR10 | runtime per image: 0.10 vs 1.6; 0.13 vs 10.8 | Helped |
| Use FLIPD instead of NB | Stable Diffusion latent-space LID estimation · LAION | runtime per image: 0.20; > 9 × 10^3 | Helped |
| Apply FLIPD with a UNet instead of an MLP | FLIPD image-complexity estimation · Natural images | LID estimate consistency | Mixed |
| Set t₀ closer to zero | FLIPD · Synthetic data | LID estimate accuracy | Helped |
| Increase t₀ | FLIPD · Natural-image PNG compression ranking | Spearman correlation with PNG compression size | Hurt |
| Use Kneedle to select t₀ with a UNet (ablation) | FLIPD · UNet-based image LID estimation | LID estimate sensibility | Hurt |
| Use Kneedle for knee-based t₀ selection (ablation) | FLIPD · Synthetic LID benchmarks | LID estimation performance | Helped |
| Use the efficient ODE-based LIDL implementation instead of FLIPD (ablation) | LID estimation with diffusion models · Synthetic benchmarks | LID estimation performance | Hurt |
Surprising findings
- Using the Kneedle knee detection with UNet backbones fails to produce sensible FLIPD LID estimates, so t0 is set as a hyperparameter instead.
- NB with the originally proposed setting consistently produces estimates almost equal to the ambient dimension.
Limitations
- FLIPD estimates on natural images are sensitive to the choice of network architecture (MLP vs UNet) and t0.
- Kneedle-based automatic t0 selection fails for UNet backbones, leaving t0 as a manually chosen hyperparameter.
- The linear-case theorem is proven only for affine subspaces; extension to non-linear manifolds is conjectured, not proven.
Experimental setup
Datasets: MNIST FMNIST SVHN CIFAR10 LAION-Aesthetics LAION-5B synthetic union-of-manifolds benchmark
Benchmarks: synthetic union-of-manifolds LID benchmark MNIST FMNIST SVHN CIFAR10 LAION-Aesthetics
Metrics: mean absolute error concordance index Spearman correlation with PNG compression size runtime in seconds
Baselines: NB LIDL ESS LPCA MLE FIS
Builds on: LIDL Normal bundle estimator score-based SDE diffusion models Hutchinson trace estimator Kneedle
Closest papers by meaning
- A Wiener Process Perspective on Local Intrinsic Dimension Estimation Methods (AAAI 2025 · similarity 0.91)
- Diffusion Models Encode the Intrinsic Dimension of Data Manifolds (ICML 2024 · similarity 0.88)
- DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents (ICML 2024 · similarity 0.87)
- Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifold (NeurIPS 2026 · similarity 0.86)
- Revisiting Diffusion Model Predictions Through Dimensionality (NeurIPS 2026 · similarity 0.86)
- CW Complex Hypothesis for Image Data (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.