Research map / Large language models / Alignment and preferences
The Alignment Game: A Theory of Long-Horizon Alignment Through Recursive Curation
Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson, Lukasz Golab
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| make Owner and Public preferences perfectly aligned in recursive BT curation | Recursive BT curation (text game, GPT-2) · Text alignment game (WikiText-2) | converged word count: converge to the shared target of 4 words within 10 iterations | Hurt |
| use partially overlapping Owner and Public preferences | Recursive BT curation (text game, GPT-2) · Text alignment game (WikiText-2) | converged word count: around 3-4 words | Mixed |
| use fully disjoint Owner and Public preferences | Recursive BT curation (text game, GPT-2) · Text alignment game (WikiText-2) | Public-stage word count: Owner range 3-4 words, Public settles around 4 words | Mixed |
Closest papers by meaning
- Direct Alignment with Heterogeneous Preferences (NeurIPS 2025 · similarity 0.90)
- Understanding the Learning Dynamics of Alignment with Human Feedback (ICML 2024 · similarity 0.90)
- The Burden of Interactive Alignment with Inconsistent Preferences (NeurIPS 2025 · similarity 0.89)
- Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences? (NeurIPS 2025 · similarity 0.89)
- Self-Consuming Generative Models with Co-Evolving Human Preferences (NeurIPS 2026 · similarity 0.88)
- AI Alignment with Changing and Influenceable Reward Functions (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
