Research map / Multi-agent systems and games / Games and equilibria
Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bench-MFG, a JAX-based benchmark suite for discrete-space stationary mean field games with a taxonomy of problem classes, a random MF-Garnet generator, and a black-box PSO exploitability minimizer (MF-PSO), benchmarking classical MFG solvers.
Problem
Mean field game algorithms are evaluated on ad-hoc, isolated, simplistic environments, lacking a standardized protocol to assess robustness, generalization, and failure modes in stationary MFGs.
Approach
A unified benchmark with a taxonomy (no-interaction, contractive, Lasry-Lions monotone, multiple-equilibria, potential, cyclic, dynamics-coupled) of prototypical environments, a procedural random-instance generator (MF-Garnets) with additive/multiplicative couplings, and a black-box particle swarm exploitability minimizer evaluated alongside FP, PI, BPI, OMD.
Key results
- On MF-Garnet 5x5x5 (A/M), FP achieves mean exploitability 1.6537 with high variance, while PSO attains 0.2250 with std 0.2115 among the best.
- Fictitious Play converges in non-monotone settings despite lacking theoretical guarantees, per the authors.
- OMD typically yields the lowest exploitability in cyclic games but is highly sensitive to hyperparameters; the JAX implementation claims ~2000x speedup over classical Python.
Surprising findings
- Fictitious Play converges in non-monotone settings where theoretical guarantees are absent.
- Boltzmann PI and OMD have high exploitability on large MF-Garnet instances (about 0.95 and 1.44 in the 25x10x10 setting), while FP-type methods reach very low values.
Limitations
- Only moderate state spaces tested; large-scale environments left to future work.
- Only stationary setting; finite-horizon MFGs not covered.
- Only model-based iterative solvers evaluated; no sample-based deep RL agents.
Experimental setup
Datasets: MF-Garnet random instances (procedurally generated)
Benchmarks: Bench-MFG environments (NI, C, LL, two beach bars, 4 rooms, RPS, SIS epidemic, kinetic congestion) MF-Garnet instances
Metrics: exploitability standard deviation of final exploitability
Baselines: Fixed Point Damped Fixed Point Fictitious Play Policy Iteration Smoothed PI Boltzmann PI Online Mirror Descent PSO
Builds on: Garnet MDP testbed Fictitious Play Online Mirror Descent Particle Swarm Optimization JAX
Closest papers by meaning
- Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation (ICLR 2025 · similarity 0.92)
- Learning Discrete-Time Major-Minor Mean Field Games (AAAI 2024 · similarity 0.92)
- Bounded Rationality Equilibrium Learning in Mean Field Games (AAAI 2025 · similarity 0.90)
- Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics (NeurIPS 2025 · similarity 0.90)
- Learning Mean Field Games on Sparse Graphs: A Hybrid Graphex Approach (ICLR 2024 · similarity 0.90)
- Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean Field Games (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.