Research map / Optimization / Nonconvex and smooth optimization
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use magnetic mirror descent for QRE solving | MMD · Normal-form and extensive-form games | convergence to QRE | Helped |
| Use behavioral-form MMD with action-value feedback | MMD · Kuhn Poker, 2x2 Abrupt Dark Hex, 4-Sided Liar's Dice, Leduc Poker | convergence to AQRE | Helped |
| Use MMD as a tabular self-play reinforcement learning algorithm | MMD · Tabular two-player zero-sum benchmarks | exploitability | Mixed |
| Use annealed temperature and stepsize with a constant magnet | MMD · Tabular two-player zero-sum benchmarks | exploitability | Helped |
| Use a moving magnet with constant temperature and stepsize | MMD · Tabular two-player zero-sum benchmarks | exploitability | Helped |
| Train for 10 million rather than 1 million time steps | MMD and NFSP · 3x3 Abrupt Dark Hex and Phantom Tic-Tac-Toe | approximate exploitability | Helped |
| Train PPO over time | PPO · 3x3 Abrupt Dark Hex and Phantom Tic-Tac-Toe | approximate exploitability | No clear effect |
| Replace PPO's adaptive forward-KL regularization with reverse-KL regularization | PPO-based MMD · 3x3 Abrupt Dark Hex and Phantom Tic-Tac-Toe | approximate exploitability | Helped |
Closest papers by meaning
- Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment (ICLR 2025 · similarity 0.89)
- Local and Adaptive Mirror Descents in Extensive-Form Games (NeurIPS 2024 · similarity 0.88)
- Adaptively Perturbed Mirror Descent for Learning in Games (ICML 2024 · similarity 0.88)
- Operator World Models for Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Learning mirror maps in policy mirror descent (ICLR 2025 · similarity 0.87)
- Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.