Research map / Learning theory and algorithms / Optimal transport
Sample and Map from a Single Convex Potential: Generation using Conjugate Moment Measures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes conjugate moment measures, a factorization rho = grad(w*) # e^{-w} with w convex, learned via fixed-point OT iterations (CMFGen) or Monge-Ampère regression from energies (CMFMA), parameterized by ICNNs.
Problem
Standard generative models fix a Gaussian noise prior and learn a separate transport map; moment measure factorizations tie sampling and mapping through one convex potential, but the resulting log-concave Gibbs factor has spread inversely related to the target, making it ill-suited for generation.
Approach
Factorize a target as rho = grad(w*) # e^{-w} (conjugate of the moment measure factorization), so the log-concave Gibbs factor follows the target's shape; estimate w by iterating OT maps between rho and the Gibbs factor (Schauder fixed point), with an ICNN parameterization and LMC sampling plus conjugate inversion.
Key results
- On 2D distributions, CMFGen gives Sinkhorn divergence equal or better than a generative ICNN trained to map Gaussian directly to the target.
- For Gaussians, the conjugate Gibbs factor is N(r, Sigma^{1/3}), whereas the moment-measure Gibbs factor is N(m, Sigma^{-1}), so the conjugate version matches target spread better.
- CMFGen is reported as the first successful ICNN generation of the MNIST distribution, with higher quality than a direct ICNN baseline on MNIST and Cartoon.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the conjugate moment factorization instead of the standard moment measure factorization | Gibbs factor for the target distribution · Univariate Gaussian mixtures ρ1 and ρ2 | similarity of Gibbs factor to target distribution | Helped |
| Use CMFGen instead of a generative ICNN trained to directly transport Gaussian noise | CMFGen · 2D distributions: Circles, S-curve, Checkerboard, Scaled-Rotated S-curve, and Diag-Checkerboard | Sinkhorn divergence | Helped |
| Use CMFGen instead of an ICNN trained to directly transport Gaussian noise | CMFGen · MNIST and Cartoon | sample quality | Helped |
| Estimate the conjugate moment potential from samples with CMFGen | CMFGen · 2D sample-defined distributions | alignment of generated and target distributions | Helped |
| Estimate the conjugate moment potential from an energy function with CMFMA | CMFMA · 2D energy-defined benchmarks | sample fidelity | Helped |
Surprising findings
- The moment-measure Gibbs factor of a peaked Gaussian is more spread out than the target, the reverse of what is desired for generative modeling.
- Samples from the log-concave noise distribution already show digit and face features before applying the conjugate map.
Limitations
- Gaussian and 1D results are preliminary; high-dimensional results are limited to MNIST and Cartoon.
- Sampling requires LMC warm-up and an iterative conjugate solver, adding inference cost.
- Existence proof assumes absolutely continuous measures on compact convex supports.
Experimental setup
Datasets: MNIST Cartoon dataset 2D synthetic distributions (Circles, S-curve, Checkerboard, Scaled-Rotated S-curve, Diag-Checkerboard) 1D Gaussian mixtures
Benchmarks: MNIST Cartoon dataset 2D synthetic distributions
Metrics: Sinkhorn divergence visual quality of samples
Baselines: moment measure potential (fixed-point) ICNN generative map from Gaussian (Makkuva et al. solver)
Builds on: moment measures neural OT solvers (Makkuva et al.) input convex neural networks Brenier's theorem
Closest papers by meaning
- One-Shot Generative Flows: Existence and Obstructions (NeurIPS 2026 · similarity 0.85)
- On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models (ICLR 2024 · similarity 0.84)
- Generative diffusion for perceptron problems: statistical physics analysis and efficient algorithms (NeurIPS 2025 · similarity 0.84)
- NETS: A Non-equilibrium Transport Sampler (ICML 2025 · similarity 0.84)
- Posterior Sampling by Combining Diffusion Models with Annealed Langevin Dynamics (NeurIPS 2025 · similarity 0.84)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.