Research map / Reinforcement learning / Policy gradient and actor-critic

Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization

ICML 2025 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an inverse game theory framework that recovers the feasible payoff/reward sets of entropy-regularized (QRE) two-player zero-sum matrix and Markov games from observed actions, with identifiability conditions and confidence-set estimators.

Problem

Recovering the unknown reward/payoff functions of two-player zero-sum matrix and Markov games from observed player strategies and actions, where the inverse problem is ill-posed and observational coverage is limited.

Approach

Uses entropy regularization so that the observed strategies are quantal response equilibria, which turns the inverse problem into a linear system in linear-parameterized payoff/reward parameters; identifiability is characterized by a rank condition, and a norm-constrained confidence set captures all feasible parameters when the rank condition fails.

Key results

• Parameter estimation error bound scales as O(1/N) in squared norm for payoff matrix Frobenius error, with O((m^2+n^2+(m+n)log(1/δ))/N).
• Confidence set converges to the feasible set in Hausdorff distance at rate O(sqrt((m^2+n^2+(m+n)log(1/δ))/N)).
• Markov game reward set Hausdorff-type error D(R, R_hat) bounded at rate O(T^{-1/2}) up to log factors.

Limitations

• Strong identifiability requires a rank condition on QRE-derived matrices that may fail with limited exploration
• Markov game guarantees require C-well-posedness state coverage assumption
• Linear parameterization of payoffs/features is assumed

Experimental setup

Datasets: synthetic observed QRE action data

Metrics: Hausdorff distance between feasible and estimated sets parameter estimation error Frobenius norm payoff error

Builds on: Maximum Entropy IRL quantal response equilibrium linear MDP

Closest papers by meaning

• On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Playing Markov Games Without Observing Payoffs (NeurIPS 2026 · similarity 0.88)
• Recent Advancements in Inverse Reinforcement Learning (AAAI 2024 · similarity 0.88)
• A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback (ICML 2024 · similarity 0.88)
• Optimism Without Regularization: Constant Regret in Zero-Sum Games (NeurIPS 2025 · similarity 0.87)
• Sub-optimal Experts mitigate Ambiguity in Inverse Reinforcement Learning (NeurIPS 2024 · similarity 0.87)