Research map / Reinforcement learning / Policy gradient and actor-critic

Toward Optimal Policy Population Growth in Two-Player Zero-Sum Games

ICLR 2024 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces anytime double oracle (ADO) and an approximate extensive-form variant (anytime PSRO) that keep exploitability from increasing between iterations of policy population growth in two-player zero-sum games.

Problem

Deep RL methods such as PSRO often increase exploitability between iterations when growing policy populations in competitive two-player zero-sum games, which is problematic for large games.

Approach

An anytime double oracle algorithm guaranteeing non-increasing exploitability across iterations, with an approximate extensive-form version (anytime PSRO) for scaling to large games.

Experimental setup

Metrics: exploitability

Baselines: PSRO

Builds on: policy space response oracles double oracle

Closest papers by meaning

• Global Policy-Space Response Oracles for Two-Player Zero-Sum Games (ICML 2026 · similarity 0.93)
• Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games (NeurIPS 2020 · similarity 0.91)
• Explicit Exploration for High-Welfare Equilibria in Game-Theoretic Multiagent Reinforcement Learning (ICML 2025 · similarity 0.89)
• A-PSRO: A Unified Strategy Learning Method with Advantage Metric for Normal-form Games (ICML 2025 · similarity 0.89)
• Learning Diverse Risk Preferences in Population-Based Self-Play (AAAI 2024 · similarity 0.88)
• A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence (ICLR 2025 · similarity 0.87)