Research map / Probabilistic and scientific modeling / Variational and generative sampling
Analysis of Learning a Flow-based Generative Model from Limited Sample Complexity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a sharp asymptotic end-to-end analysis of a flow-based generative model with a shallow two-layer denoising autoencoder trained on finite samples from a high-dimensional Gaussian mixture, showing the generated mean error decays as Θ(1/n), which is Bayes-optimal.
Problem
Understanding how a flow-based generative model learned from a finite number of training samples behaves, including overfitting/memorization effects on the generated density, in high dimensions.
Approach
Closed-form asymptotic characterization of the learnt velocity field of a two-layer DAE with a trainable skip connection trained per time step on a finite sample of a binary Gaussian mixture, combined with summary-statistic ODEs tracking the generative flow; shows the generated mean's squared error decays as Θ(1/n) matching the Bayes-optimal rate.
Key results
- The squared distance between the generated mixture mean and the target mean decays as Θ_n(1/n), which the paper states is Bayes-optimal (Bayes-optimal MSE σ²/(n+σ²)).
- Closed-form predictions for the learnt skip strength, weight overlaps, and summary-statistic flow agree with numerical simulations in d=500 with Adam training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the shallow DAE-parametrized flow from n samples | DAE-parametrized learned flow · Binary Gaussian mixture | generated cluster mean MSE: \Theta_n(\sfrac{1}{n}) | Helped |
| Use the Bayes-optimal estimator with known mixture form and variance | Bayes-optimal estimator · Binary Gaussian mixture | cluster mean MSE: \frac{\sigma^2}{n+\sigma^2}; \Theta_n(\sfrac{1}{n}) | Helped |
| Train the shallow DAE-parametrized flow from n samples | DAE-parametrized learned flow · Binary Gaussian mixture | cosine dissimilarity of generated cluster mean: \Theta_n(\sfrac{1}{n}) | Helped |
Surprising findings
- The learnt weights retain a non-zero component along the training-sample aggregate vector ξ, signalling overfitting, and this overfitting is non-monotonic in time.
Limitations
- Analysis restricted to a binary balanced isotropic Gaussian mixture in the high-dimensional limit with a shallow one-hidden-neuron DAE.
- A separate network is trained per time step, unlike practical time-conditioned networks.
- Results are heuristic (replica-style partition function computation) rather than rigorous proofs.
Experimental setup
Datasets: synthetic binary Gaussian mixture
Benchmarks: synthetic Gaussian mixture
Metrics: mean squared error of cluster mean cosine similarity denoiser MSE
Baselines: PCA estimator of cluster mean Bayes-optimal estimator oracle denoiser
Builds on: stochastic interpolation denoising autoencoder Tweedie's formula
Closest papers by meaning
- Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization (ICLR 2024 · similarity 0.90)
- A solvable model of learning generative diffusion: theory and insights (NeurIPS 2025 · similarity 0.89)
- Mean Flows for One-step Generative Modeling (NeurIPS 2025 · similarity 0.88)
- Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks (ICLR 2025 · similarity 0.87)
- Expanding Flow Maps (NeurIPS 2026 · similarity 0.87)
- Gaussian Mixture Flow Matching Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.