Research map / Reinforcement learning / Policy gradient and actor-critic
A Natural Actor-Critic Framework for Zero-Sum Markov Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use natural actor-critic (softmax/KL update) with FoRB stage 1 and NPG stage 2 instead of Euclidean projected PG, under Assumption 2.1 and 3.3 | two-player zero-sum Markov games (tabular) · theoretical sample complexity | sample complexity to epsilon-Nash | Helped |
| Add ζ-greedy exploration and drop the uniform policy lower bound (Assumption 3.3) | Reflected NAC · theoretical sample complexity | sample complexity to epsilon-Nash: ϵ^-4 instead of ϵ^-2 | Hurt |
| Use fresh TD(0) value estimates at every inner iteration of stage 1 rather than reusing a stage-2 estimate | Reflected NAC stage 1 · theoretical analysis | bias bound on value estimate | Helped |
| Numerical: Reflected NAC vs OGDA and REINFORCE on bandits and Alesia | Reflected NAC · 100-arm bandit and Alesia (L=3, C=6) | convergence speed (probability of optimal action / suboptimality gap) | Helped |
Closest papers by meaning
- Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence beyond the Minty Property (AAAI 2024 · similarity 0.92)
- Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games (ICLR 2023 · similarity 0.92)
- Policy Optimization for Markov Games: Unified Framework and Faster Convergence (NeurIPS 2022 · similarity 0.92)
- Decentralized Single-Timescale Actor-Critic on Zero-Sum Two-Player Stochastic Games (ICML 2021 · similarity 0.92)
- Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedback (NeurIPS 2023 · similarity 0.91)
- Solving Zero-Sum Convex Markov Games (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.