Research map / Image generation / Fast diffusion sampling
What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies local geometry of pretrained generative models (DDPM, DiT, Stable Diffusion) via three CPWL-derived descriptors (local scaling, rank, complexity) and shows they correlate with aesthetics, diversity and memorization, then uses a reward model on local scaling for geometry-guided denoising.
Problem
Generation quality, diversity and memorization vary across the learned data manifold of deep generative models, and it is unclear how this relates to measurable properties of the generator's local geometry.
Approach
Applies continuous piecewise-linear (CPWL) generator theory to define three local descriptors (log-volume scaling psi, local rank nu, local complexity delta) computed from input-output Jacobians with randomized SVD, links local scaling to negative log-likelihood, and trains a reward model on local scaling to steer denoising.
Key results
- Reward guidance trained on only 50K samples raised local scaling, Vendi diversity and aesthetic scores for most pre-guidance bins of Stable Diffusion.
- Local scaling and complexity track visual complexity and diversity; diversity rises with local scaling then drops at the highest bins (anti-modes).
- Memorized prompts and higher classifier-free guidance produce distinctly lower local scaling and rank trajectories than COCO prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| maximize local scaling with reward-model guidance | Stable Diffusion generation · Imagewoof dog prompts | Vendi score | Helped |
| maximize local scaling with reward-model guidance | Stable Diffusion generation · Imagewoof dog prompts | aesthetic score | Helped |
| maximize local scaling with reward-model guidance | Stable Diffusion generation · Imagewoof dog prompts | local scaling | Helped |
| minimize local scaling with reward-model guidance | Stable Diffusion generation · Stable Diffusion denoising | visual detail | Hurt |
| increase local scaling level set | Stable Diffusion generated ImageNet images · ImageNet | Vendi score | Mixed |
| increase corruption severity for spectral-band or color-range reduction distortions | Stable Diffusion decoder local geometry · ImageNet corrupted images | local scaling | Hurt |
| apply distortions that introduce high-frequency artifacts | Stable Diffusion decoder local geometry · ImageNet corrupted images | local scaling | Hurt |
| increase classifier-free guidance scale | Stable Diffusion denoising · Stable Diffusion generation | local scaling | Hurt |
| increase additive training-noise standard deviation | VAE · MNIST | local complexity | Helped |
| increase additive training-noise standard deviation | VAE · MNIST | local scaling | Hurt |
| increase local scaling level set | DiT-XL · ImageNet-256 / Imagewoof | Vendi score | Mixed |
Surprising findings
- Highest local scaling bins are anti-modes with lower diversity, yet real-image aesthetic and artifact scores rise again in the highest local scaling bins.
- Beta-VAE training shows a double ascent in local scaling rather than the double descent seen in local complexity for classifiers.
Limitations
- Local scaling requires input-output Jacobian computation, which is expensive for large models.
- Local geometry depends on training dynamics, so comparing separately trained models requires care.
Experimental setup
Datasets: ImageNet Imagewoof COCO MNIST
Benchmarks: ImageNet COCO captions Dreambooth prompts with Imagewoof subjects
Metrics: local scaling Vendi score RAHF aesthetic score artifact score
Baselines: classifier-free guidance baseline sampling (Stable Diffusion without reward guid
Builds on: continuous piecewise-linear generator theory SplineCAM classifier-free guidance universal guidance
Closest papers by meaning
- Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion (ICLR 2025 · similarity 0.89)
- Distance Marching for Generative Modeling (NeurIPS 2026 · similarity 0.88)
- A Geometric Framework for Understanding Memorization in Generative Models (ICLR 2025 · similarity 0.87)
- Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes (ICML 2024 · similarity 0.86)
- Harmonizing Geometry and Uncertainty: Diffusion with Hyperspheres (ICML 2025 · similarity 0.86)
- GPLD3D: Latent Diffusion of 3D Shape Generative Models by Enforcing Geometric and Physical Priors (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.