Research map / Multi-agent systems and games / Games and equilibria
LOPT: Learning Optimal Pigovian Tax in Sequential Social Dilemmas
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces externality theory into mixed-motive MARL and proposes LOPT, where a centralized tax planner learns Pigovian tax/allowance rates that reshape each agent's reward to resolve social dilemmas in Escape Room and Cleanup.
Problem
Social dilemmas in mixed-motive multi-agent RL, where self-interested agents' individual rewards diverge from social welfare; the paper seeks a numerical way to quantify these dilemmas and align incentives.
Approach
Extends economic externalities to MARL (influence of one agent's actions on others' rewards not reflected in its local reward) and uses a centralized tax planner trained to maximize social welfare to output per-agent tax and allowance rates, which shape individual rewards toward the optimal Pigovian tax reward shaping.
Key results
- In Escape Room ER(N=2,M=1) and ER(N=3,M=2), LOPT converges to the optimal values of 8 and 9 respectively and completes in one step without betrayal.
- In Cleanup(N=2), LOPT achieves near-optimal social welfare on 7x7 and 10x10 maps, while LIO fails to learn efficient policies.
- LOPT scales to Cleanup(N=5) on an 18x25 map, where it is reported to outperform PPO, SCM and MOA baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LOPT's learned Pigovian tax and allowance reward shaping | Escape Room (N=2, M=1) · Escape Room (N=2, M=1) | social welfare | Helped |
| use LOPT's learned Pigovian tax and allowance reward shaping | Escape Room (N=3, M=2) · Escape Room (N=3, M=2) | social welfare | Helped |
| use LOPT's learned Pigovian tax and allowance reward shaping | Escape Room · Escape Room | steps to end episode: 1 step | Helped |
| replace selfish Policy Gradient with LOPT | Escape Room · Escape Room | social welfare | Helped |
| replace LIO or LIO-dec with LOPT | Escape Room · Escape Room | learning stability | Helped |
| use LOPT's learned Pigovian tax and allowance reward shaping | Cleanup (N=2) · Cleanup (N=2), 7×7 and 10×10 maps | social welfare | Helped |
| replace LIO with LOPT | Cleanup (N=2) · Cleanup (N=2), 7×7 and 10×10 maps | social welfare | Helped |
| scale the map from 7×7 to 10×10 | AC-d · Cleanup (N=2) | social welfare | Hurt |
| scale the map from 7×7 to 10×10 | IA and AC-c · Cleanup (N=2) | social welfare | Hurt |
| keep agent orientations fixed | LOPT · Cleanup (N=2), 10×10 map | social welfare | No clear effect |
| scale to five agents on an 18×25 map | LOPT · Cleanup (N=5), 18×25 map | social welfare | Helped |
Limitations
- Centralized tax planner raises computational complexity and scalability concerns.
- Theoretical equivalence to optimal Pigovian tax holds only when other agents' interactive influence is ignored (Theorem 2 assumption).
Experimental setup
Datasets: Escape Room Cleanup
Benchmarks: Escape Room Cleanup
Metrics: social welfare (cumulative reward) episode steps to completion betrayal rate
Baselines: LIO LIO-dec Inequity Averse Model of Other Agents Social Curiosity Module PPO Actor-Critic Policy Gradient
Builds on: LIO AI Economist Pigovian tax
Closest papers by meaning
- Self-Play $Q$-Learners Can Provably Collude in the Iterated Prisoner's Dilemma (ICML 2025 · similarity 0.85)
- Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning (ICML 2025 · similarity 0.84)
- Aligning Individual and Collective Objectives in Multi-Agent Cooperation (NeurIPS 2024 · similarity 0.84)
- Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games (NeurIPS 2024 · similarity 0.84)
- Training Generalizable Collaborative Agents via Strategic Risk Aversion (NeurIPS 2026 · similarity 0.84)
- Learning to Mitigate Externalities: the Coase Theorem with Hindsight Rationality (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.