Research map / Probabilistic and scientific modeling / Variational and generative sampling
Constrained Sampling with Primal-Dual Langevin Monte Carlo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes primal-dual Langevin Monte Carlo (PD-LMC), a single-particle stochastic gradient descent-ascent in Wasserstein space that samples from a target distribution under moment/expectation constraints, with convergence analysis for convex and LSI settings.
Problem
Sampling from a distribution known up to normalization while satisfying statistical constraints on expectations of nonlinear functions (e.g., fairness, moment constraints), which support-constraint methods (projection, mirror, barrier) cannot handle.
Approach
Runs Langevin dynamics on the Lagrangian jointly with dual ascent updates on multipliers using single-sample constraint evaluations, i.e., a stochastic Wasserstein gradient descent-ascent that avoids explicit expectation integration, and analyzes its convergence with KL/W2 rates.
Key results
- On 1D truncated Gaussian, PD-LMC gives mean 1.508 vs true 1.510, versus 1.488 for Proj. LMC and 1.470 for Mirror LMC.
- On Adult fairness constraint (delta=0.01), constrained PD-LMC reduces positive-prediction rates to 18.1% (male) and 15.1% (female) with accuracy 82% vs 84% unconstrained.
- On the 2D ball-restricted Gaussian, Proj. LMC places almost 25% of samples on the boundary while PD-LMC keeps only 1.8% outside the support.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PD-LMC instead of Projected LMC | Sampling from a 1D Gaussian truncated to [1,3] · 1D truncated Gaussian [1,3] | mean estimate: 1.488 -> 1.508 | Helped |
| Use PD-LMC instead of Mirror LMC | Sampling from a 1D Gaussian truncated to [1,3] · 1D truncated Gaussian [1,3] | mean estimate: 1.470 -> 1.508 | Helped |
| Use PD-LMC instead of Projected LMC | Sampling from a 2D Gaussian restricted to a unit ℓ2-norm ball · 2D truncated Gaussian, unit ℓ2-norm ball | boundary sample rate: almost 25% -> 0.14% expected | Helped |
| Use PD-LMC instead of Mirror LMC | Sampling from a 2D Gaussian restricted to a unit ℓ2-norm ball · 2D truncated Gaussian, unit ℓ2-norm ball | mean estimate: Mirror LMC: (0.312,0.418) vs. PD-LMC: (0.446,0.444) vs. true mean: (0.368,0.368) | Mixed |
| Constrain male and female positive-outcome rates with δ=0.01 | Bayesian logistic regression on Adult · Adult test set | gender disparity in positive prediction rates: unconstrained: 26.2% and 5% -> constrained: 18.1% and 15.1% | Helped |
| Add gender rate constraints with δ=0.01 | Bayesian logistic regression on Adult · Adult test set | accuracy: 84% -> 82% | Hurt |
| Use mini-batches for the dual updates | PD-LMC · Constrained sampling experiments | performance | No clear effect |
Surprising findings
- Mini-batches increase computational cost with no performance benefit in the experiments.
- PD-LMC can produce samples outside the constraint set, though this becomes less frequent over iterations, since constraints apply to the distribution rather than samples.
Limitations
- Samples may fall outside the constraint set; PD-LMC is not an interior-point method.
- LSI-based variant requires multiple inner LMC iterations per dual update, increasing cost substantially.
- Equality constraints are not covered by the LSI analysis; the bounded gradient and variance assumptions are required along iterates.
Experimental setup
Datasets: Adult
Benchmarks: truncated Gaussian sampling Adult fairness (Bayesian logistic regression) Bayesian stock market counterfactual
Metrics: sample mean fraction of samples outside constraint set positive prediction rate by gender accuracy
Baselines: Projected LMC Mirror LMC Dual ascent LMC
Builds on: Langevin Monte Carlo gradient descent-ascent dual ascent sampling (Chamon et al.) projected LMC mirror LMC
Closest papers by meaning
- Constrained Langevin Algorithms with L-mixing External Random Variables (NeurIPS 2022 · similarity 0.90)
- Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing (NeurIPS 2025 · similarity 0.89)
- Non-asymptotic Error Bounds in $\mathcal{W}_2$-Distance with Sqrt(d) Dimension Dependence and First Order Convergence for Langevin Monte Carlo beyond Log-Concavity (ICML 2025 · similarity 0.88)
- Langevin Monte Carlo for strongly log-concave distributions: Randomized midpoint revisited (ICLR 2024 · similarity 0.87)
- Functional Gradient Flows for Constrained Sampling (NeurIPS 2024 · similarity 0.87)
- Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.