Research map / Optimization / Nonconvex and smooth optimization
Convergence of Mean-Field Langevin Stochastic Descent-Ascent for Distributional Minimax Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes discrete-time mean-field Langevin stochastic descent-ascent (MFL-SDA) for distributional minimax problems, proving an O(1/ε log 1/ε) last-iterate complexity via an elementary perturbation analysis instead of PDE techniques.
Problem
Last-iterate convergence rate of discrete-time single-loop mean-field Langevin stochastic descent-ascent for distributional minimax optimization, where prior discrete-time analyses were suboptimal or double-loop.
Approach
A direct discrete-time Lyapunov analysis (not via continuous-time gradient flow or PDEs) of alternating stochastic Wasserstein gradient descent-ascent with Gaussian noise, using integration by parts and the divergence theorem to control Taylor higher-order terms, under a two-sided distributional log-Sobolev (PL-type) condition.
Key results
- MFL-SDA achieves last-iterate complexity K = O(d/(ε τ^4 α^4) log 1/ε) (sample complexity), with geometric decay to a bias term R2 = O(η1) under the stated assumptions.
- Euclidean-matching rate: the O(1/ε log 1/ε) rate is claimed nearly optimal relative to the O(1/ε) Euclidean SGDA rate under two-sided PL.
- Corollary gives KL and W2 convergence to the saddle point, claimed to improve the complexity of stochastic MFL-AG when the LSI constant α is small.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use discrete-time MFL-SDA with stochastic gradient oracles | Distributional minimax optimization · Distributional minimax optimization | last-iterate iteration complexity: O( 1/ϵ log 1/ϵ ) | Helped |
| use MFL-SDA instead of MFL-AG | Distributional minimax optimization · Distributional minimax optimization | convergence complexity: O( 1/ϵ log 1/ϵ ); O(ϵ−1−O(1/α)) | Helped |
| use MFL-SDA instead of MFL-ABR | Distributional minimax optimization · Distributional minimax optimization | iteration complexity: O( 1/ϵ log 1/ϵ ) | No clear effect |
| use MFL-SDA instead of MFLD | Distributional minimax optimization · Distributional minimax optimization | convergence complexity: O( 1/ϵ log 1/ϵ ) | No clear effect |
| use MFL-SDA instead of stochastic gradient descent-ascent | Distributional minimax optimization · Euclidean minimax optimization | convergence complexity: O( 1/ϵ log 1/ϵ ); O( 1/ϵ ) | Mixed |
Surprising findings
- The paper notes that the bias term from Gaussian noise prevents absorbing higher-order Taylor terms, so the discrete-time distributional rate is sublinear, unlike the Euclidean case with linear rate.
Limitations
- Results depend on strong assumptions (bounded derivatives up to fourth order, bounded cross variations, LSI constant) and the LSI constant may be very conservative (e.g., O(1/d) or exponential in τ^-1).
- Analysis uses infinite particles; finite-particle approximation and uniform-in-time propagation of chaos are not addressed.
- Two-timescale step-size ratio is large, and the complexity has high-order dependence on τ and α (τ^-4 α^-4).
Experimental setup
Metrics: iteration complexity Lyapunov function decay KL divergence Wasserstein-2 distance
Baselines: MFL-AG (Kim et al., 2024) MFL-ABR (Kim et al., 2024) Lu (2023) MFL-DA Euclidean SGDA (Yang et al., 2020)
Builds on: Mean-field Langevin dynamics (Nitanda et al., 2022a) Yang et al. 2020 alternating stochastic gradient descent-ascent under PL Lu 2023 two-sided PL MFL-DA Kim et al. 2024 MFL-AG/MFL-ABR
Closest papers by meaning
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR 2024 · similarity 0.91)
- High-probability complexity bounds for stochastic non-convex minimax optimization (NeurIPS 2024 · similarity 0.88)
- Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems (NeurIPS 2026 · similarity 0.87)
- Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems (NeurIPS 2026 · similarity 0.87)
- Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods (ICLR 2024 · similarity 0.87)
- Mean Field Langevin Actor-Critic: Faster Convergence and Global Optimality beyond Lazy Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.