Research map / Reinforcement learning / MDPs and planning
Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove the state-visitation factor from the policy update | Independent policy gradient for Markov potential games · Markov potential games | iteration complexity: O(1/\epsilon^2) | Helped |
| Use linear function approximation and sample-based value estimates | Independent policy gradient for Markov potential games · Markov potential games with potentially infinitely large state spaces | sample complexity: O\left(\frac{1}{\epsilon^{5}}\right) | Helped |
| Use the cooperative-game setting with identical rewards | Independent policy gradient · Markov cooperative games | iteration complexity: \dfrac{\tilde{\kappa}_\rho\, AN}{(1-\gamma)^4\, \epsilon^2} | Helped |
| Add optimistic updates and a smoothed critic | Independent policy gradient · Two-player Markov cooperative and zero-sum games | convergence: O(1/\epsilon^6) | Helped |
| Use the same optimistic algorithm across cooperative and zero-sum game types | Optimistic independent policy gradient with smoothed critic · Two-player Markov cooperative and zero-sum games | last-iterate convergence | Helped |
| Use a large stepsize with the proposed independent policy gradient | Independent policy gradient · MDP where every state defines a congestion game | convergence | Helped |
| Use projected stochastic gradient ascent with stepsize η ≥ 0.001 | Projected stochastic gradient ascent · MDP where every state defines a congestion game | convergence | Hurt |
Closest papers by meaning
- Provably Fast Convergence of Independent Natural Policy Gradient for Markov Potential Games (NeurIPS 2023 · similarity 0.95)
- Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence beyond the Minty Property (AAAI 2024 · similarity 0.93)
- Solving Zero-Sum Convex Markov Games (ICML 2025 · similarity 0.92)
- Learning Nash Equilibrium of Markov Potential Games with a Shared Constraint via Primal-Dual Optimization (AAAI 2025 · similarity 0.91)
- Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics (NeurIPS 2026 · similarity 0.91)
- A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.