Research map / Probabilistic and scientific modeling / Variational and generative sampling
When and how can inexact generative models still sample from the data manifold?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why generative models (score-based and flow-matching) keep samples on the data support despite learned-drift errors, using a random dynamical systems and finite-time Lyapunov vector analysis, with a sufficient alignment condition.
Problem
Learning errors in the score or drift of dynamical generative models are inevitable, yet generated samples stay near the data support; the paper asks when and how this robustness of the support holds.
Approach
Treats the generating process as a random dynamical system and shows infinitesimal drift errors only change the predicted density on the data manifold; robustness follows when the top finite-time Lyapunov vectors align with the tangent space of the data manifold, with a sufficient dynamical condition (attracting drift with small cross-derivatives) guaranteeing this alignment.
Key results
- Theoretical result: infinitesimal score/drift errors change the predicted density only on the data manifold support, shown via a linear response formula.
- Theorem gives sufficient conditions (expansive phase, uniform attraction at end, small cross-derivatives) under which top Lyapunov vectors align with the data manifold tangent space.
- On MNIST with a pretrained SGM, adding even large-norm perturbations along leading Lyapunov vectors yields recognizable digits, while perturbations along the 100th LV leave the manifold.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add large score perturbations | Two-moons score-based generative model · Two moons | support robustness | Helped |
| Use flow matching with an 8 Gaussian source density | Flow-matching generative model · Two-dimensional examples | support robustness | Hurt |
| Add large-norm perturbations | MNIST score-based generative model · MNIST | image quality | No clear effect |
| Perturb a generated image along the most sensitive Lyapunov vector | MNIST score-based generative model · MNIST | image quality | No clear effect |
| Perturb a generated image along the 100th most sensitive Lyapunov vector | MNIST score-based generative model · MNIST | manifold adherence | Hurt |
Surprising findings
- Large errors in the score do not move generated samples off the support; the density shifts tangentially along the manifold instead.
- Flow matching with an 8-Gaussian source did not show the robustness property.
Limitations
- Only a sufficient, not necessary, condition for alignment and robustness is provided.
- Empirical validation is limited to low-dimensional toy examples and a single MNIST SGM.
- Does not address memorization or generalization.
Experimental setup
Datasets: MNIST CIFAR10 two-moons (synthetic) 8 Gaussians (synthetic)
Benchmarks: two-moons toy density MNIST
Metrics: finite-time Lyapunov exponents kernel density estimate of generated density likelihood score (qualitative)
Builds on: score-based generative models conditional flow matching Oseledets multiplicative ergodic theorem
Closest papers by meaning
- Temporal Alignment Guidance: On-manifold Sampling in Diffusion Models (NeurIPS 2026 · similarity 0.87)
- Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion (ICLR 2025 · similarity 0.87)
- Score-based generative models are provably robust: an uncertainty quantification perspective (NeurIPS 2024 · similarity 0.87)
- Sampling is as easy as keeping the consistency: convergence guarantee for Consistency Models (ICML 2024 · similarity 0.86)
- What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models (ICLR 2025 · similarity 0.86)
- Stochastic Interpolants with Data-Dependent Couplings (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.