Research map / Optimization / Nonconvex and smooth optimization
Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemiSGD, a single-loop stochastic semi-gradient method that updates the value function and population estimate jointly as one parameter, extended with population-aware linear function approximation for MFGs on continuous spaces, with finite-time convergence analysis.
Problem
Learning mean field equilibria in MFGs with fixed-point iteration is computationally heavy, oscillatory, and cannot easily use function approximation for the population measure, especially on continuous state-action spaces.
Approach
Treat the value function and population measure as a unified parameter updated simultaneously with the same online samples (SemiSGD), and apply linear function approximation to both via linear MFGs and population-aware LFA, with the concatenated semi-gradient giving a descent direction toward the MFE.
Key results
- On the speed control game, SemiSGD converges faster, is more stable without stabilization mechanisms, and reaches lower MSE than FPI variants (reported qualitatively in the main text).
- PA-LFA vs discretization on the table: PA-LFA achieves lower error across dimensions, e.g., 0.17 vs 0.20 at dim 2 and 0.06 vs 0.18 at dim 25.
- Finite-time sample complexity of O(epsilon^-2 log epsilon^-1) for SemiSGD with LFA under a small-Lipschitz contractivity condition.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 2 | MSE: $.20\pm .01$ -> $.17\pm .00$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 5 | MSE: $.21\pm .01$ -> $.15\pm .01$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 8 | MSE: $.21\pm .14$ -> $.11\pm .05$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 10 | MSE: $.27\pm .08$ -> $.12\pm .03$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 20 | MSE: $.15\pm .13$ -> $.07\pm .04$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 25 | MSE: $.18\pm .15$ -> $.06\pm .04$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 40 | MSE: $.15\pm .13$ -> $.10\pm .05$ | Helped |
| Use PA-LFA instead of discretization | Population learning · LFA vs discretization, dimension 50 | MSE: $.16\pm .13$ -> $.11\pm .02$ | Helped |
Surprising findings
- Neither the policy nor the population alone is guaranteed to follow a descent direction, even with small Lipschitz constants, while the concatenated parameter is.
Limitations
- Convergence to the exact MFE requires a contractivity-type condition (small Lipschitz constants and regularization); otherwise only a neighborhood is reached.
- For non-linear MFGs, the method converges to a projected MFE with error depending on the inherent approximation error of the basis.
Experimental setup
Benchmarks: speed control ring road MFG flocking MFG network routing MFG
Metrics: mean squared error of population distribution exploitability accuracy vs discretization
Baselines: online FPI FPI with entropy regularization FPI with fictitious play FPI with mirror descent discretization
Builds on: fixed-point iteration for MFGs linear MDP TD learning SARSA fictitious play mirror descent
Closest papers by meaning
- Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games (NeurIPS 2026 · similarity 0.92)
- Learning Discrete-Time Major-Minor Mean Field Games (AAAI 2024 · similarity 0.90)
- Bounded Rationality Equilibrium Learning in Mean Field Games (AAAI 2025 · similarity 0.90)
- Learning in Stackelberg Mean Field Games: A Non-Asymptotic Analysis (NeurIPS 2025 · similarity 0.90)
- Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics (NeurIPS 2025 · similarity 0.89)
- Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean Field Games (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.