Research map / Optimization / Nonconvex and smooth optimization
Last Iterate Convergence in Monotone Mean Field Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows last-iterate convergence of a KL proximal-point update for non-strictly monotone mean-field games, proves exponential last-iterate convergence of regularized mirror descent, and proposes APP, which approximates PP by a few RMD steps per base-policy update.
Problem
Computing equilibria of Lasry-Lions monotone mean-field games with last-iterate convergence (without time averaging) under non-strict monotonicity, where existing methods need strict monotonicity, contraction, or only average-iterate convergence.
Approach
Uses a KL-proximal-point update on the policy, shows each PP step equals a KL-regularized MFG equilibrium, and solves that subproblem with regularized mirror descent, which converges exponentially; APP alternates a few RMD steps with base-policy updates to approximate the unregularized equilibrium.
Key results
- Proves the PP update's last iterate converges to the MFG equilibrium set under non-strict monotonicity, using a Lojasiewicz inequality argument.
- Proves regularized mirror descent (RMD) converges exponentially: D(varpi*, pi^{t+1}) <= (1 - lambda*eta/2) D(varpi*, pi^t) for eta below a threshold eta*.
- Numerically on the Beach Bar Process, APP reduces exploitability with each base update and converges to the unregularized equilibrium without time averaging.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Update the base policy σ after each batch of RMD steps (APP) | RMD on the Beach Bar Process · Beach Bar Process | exploitability | Helped |
| Use APP instead of RMD | RMD on the Beach Bar Process · Beach Bar Process | equilibrium convergence | Helped |
Surprising findings
- Plain RMD can converge to a regularized equilibrium in the interior of the simplex, rather than the vertices of the unregularized equilibrium on the Beach Bar benchmark.
Limitations
- Model-based setting only; transitions and rewards assumed known, no sample complexity analysis.
- LIC for the practical APP algorithm is not proven; only conjectured, and the step-size bound eta* can be small.
- Analysis excludes mean-field-dependent transitions and no neural function approximation experiments.
Experimental setup
Datasets: Beach Bar Process
Benchmarks: Beach Bar Process
Metrics: exploitability KL divergence to equilibrium
Baselines: Online mirror descent (Perolat et al. 2022) Regularized mirror descent (Zhang et al. 2023) Fictitious play
Builds on: Mirror Descent for MFGs (Perolat et al. 2022) Regularized mirror descent for regularized MFGs (Zhang et al. 2023) Lojasiewicz inequality Beach Bar Process (Perrin et al. 2020)
Closest papers by meaning
- Last-iterate Convergence in Regularized Graphon Mean Field Game (AAAI 2025 · similarity 0.93)
- Monotone Inclusion Approach to Weakly Monotone Discrete-Time Finite-Horizon Mean-Field Games (NeurIPS 2026 · similarity 0.90)
- Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation (ICLR 2025 · similarity 0.89)
- Bounded Rationality Equilibrium Learning in Mean Field Games (AAAI 2025 · similarity 0.88)
- Near-Optimal Last-Iterate Convergence for Zero-Sum Games with Bandit Feedback and Opponent Actions (NeurIPS 2026 · similarity 0.88)
- From Average-Iterate to Last-Iterate Convergence in Games: A Reduction and Its Applications (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.