Research map / Reinforcement learning / Visual and representation RL

Position: Social Environment Design Should be Further Developed for AI-based Policy-Making

ICML 2024 · Reinforcement learning › Visual and representation RL · approach: Inferring decision-making from behavior

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper proposing Social Environment Design, a framework that uses voting on social welfare objectives plus a principal policy-maker operating in a POMG-based economy, illustrated with a sequential social dilemma apple-picking game and a list of open problems.

Problem

AI-based economic and government policy-making struggles with misaligned policy-makers, limited foresight of policy outcomes, and the lack of a general framework linking simulation, RL, and social choice.

Approach

Combines voting on policy objectives with a principal (environment designer) who repeatedly solves a Stackelberg game against followers in a parameterized, online Partially Observable Markov Game, generalizing mechanism design to dynamic economic environments.

Key results

• The paper explicitly reports no experimental results; the Apple Picking Game is presented only as a motivating example.
• Defines Social Environment Design Game and (epsilon, delta)-strong Stackelberg-Markov-Nash equilibrium formalism for repeated voting rounds.
• Lists open problems across preference aggregation, human behavior modeling, AI governance, convergence, and scaling.

Limitations

• No experimental results are reported for the Apple Picking Game or the framework.
• Many listed challenges (human behavior modeling, scaling laws) are unresolved open problems.
• Voting mechanism incentive compatibility is left to future work.

Experimental setup

Datasets: Melting Pot 2.0 apple picking environment (simulated)

Builds on: AI Economist Human Centered Mechanism Design Stackelberg-POMDP Melting Pot 2.0 Harvest game

Closest papers by meaning

• Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback (ICML 2024 · similarity 0.85)
• EconGym: A Scalable AI Testbed with Diverse Economic Tasks (NeurIPS 2025 · similarity 0.84)
• Towards a More Burkean Approach to Computational Social Choice (AAAI 2024 · similarity 0.84)
• Can an AI Agent Safely Run a Government? Existence of Probably Approximately Aligned Policies (NeurIPS 2024 · similarity 0.83)
• The Relative Value of Prediction in Algorithmic Decision Making (ICML 2024 · similarity 0.83)
• Policy Aggregation (NeurIPS 2024 · similarity 0.83)