Research map / Reinforcement learning / Policy gradient and actor-critic
Recurrent Structural Policy Gradient for Partially Observable Mean Field Games
Clarisse Wibault, Johannes Forkel, Sebastian Rene Towers, Tiphaine Wibault, Juan Agustin Duque, George Whittle, Andreas Schaab, Yucheng Yang, Chiyuan Wang, Michael A Osborne, Benjamin Moll, Jakob Nicolaus Foerster
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use history-aware recurrent policy (RSPG) instead of memoryless SPG (ablation) | Hybrid structural policy gradient · Beach Bar and Linear Quadratic (partially observable) | exploitability | Improved |
| use history-aware recurrent policy (RSPG) instead of memoryless SPG (ablation) | Hybrid structural policy gradient · Macroeconomics | exploitability | No clear effect |
| use history-aware recurrent policy (RSPG) instead of memoryless SPG (ablation) | Hybrid structural policy gradient · Macroeconomics | anticipatory behaviour | Improved |
| add recurrence over public observation history (GRU) to SPG (ablation) | SPG · convergence speed vs wall-clock | convergence rate | No clear effect |
| use known transition dynamics (HSM) instead of sample-based RL | mean-field policy learning · Linear Quadratic, Beach Bar, Macroeconomics | wall-clock convergence: approximately an order-of-magnitude faster | Improved |
| add recurrence to IPPO (RIPPO) | IPPO · training wall-clock time | training speed | Hurt |
| use M-OMD (Q-function based) in large state-action space | M-OMD · Macroeconomics | exploitability | Hurt |
| parameterise underlying continuous distribution with softmax over density on action grid instead of categorical logits (ablation) | RSPG policy · Macroeconomics | exploitability | Improved |
| compute mean-field updates in functional form rather than matrix multiplication | MFAX · Linear Quadratic mean-field update | update time: over 10x and 1000x faster than OpenSpiel and MFGLib | Improved |
Closest papers by meaning
- Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation (ICLR 2025 · similarity 0.90)
- Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games (NeurIPS 2026 · similarity 0.90)
- Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL (ICML 2024 · similarity 0.88)
- Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics (NeurIPS 2025 · similarity 0.88)
- Learning Discrete-Time Major-Minor Mean Field Games (AAAI 2024 · similarity 0.88)
- Learning in Stackelberg Mean Field Games: A Non-Asymptotic Analysis (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
