Research map / Learning theory and algorithms / Submodular and game algorithms
Scaling Laws For Scalable Oversight
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models oversight as a game between capability-mismatched Houdini and Guard players, fits piecewise-linear (double-ReLU) domain-vs-general Elo scaling across Nim, Mafia, Debate, Backdoor Code and Wargames, and derives optimal numbers of nested scalable oversight steps.
Problem
How scalable oversight performance scales with the general capability gap between the overseer (Guard) and the overseen system (Houdini), and how many nested oversight steps are optimal.
Approach
Oversight games get domain-specific Elo scores that are a piecewise-linear (double-ReLU) function of general Elo, with plateaus for task incompetence and saturation; the resulting win-probability model is used to compute the optimal number of Nested Scalable Oversight steps.
Key results
- NSO success rate at a general Elo gap of 400 is 13.5% for Mafia, 51.7% for Debate, 10.0% for Backdoor Code, and 9.4% for Wargames.
- Debate Guard (judge) domain Elo scales with general Elo while Houdini Elo stays roughly flat; Backdoor Code Houdini slope is steeper than Guard slope.
- Optimal NSO step count n* grows monotonically with general Elo gap for games where Guard slope exceeds Houdini slope; asymptotic analysis shows n* -> Δgeneral·log(10)/400 when Δdomain -> -inf.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiple oversight steps instead of a single step | Nested Scalable Oversight · Theoretical oversight games | oversight success probability | Helped |
| Increase the general Elo gap | Nested Scalable Oversight · Mafia, Debate, Backdoor Code, and Wargames | oversight success probability | Hurt |
| Increase Houdini capability relative to Guard capability | Single-step oversight games · Backdoor Code | domain Elo | Mixed |
| Increase general intelligence | Debate judges (Guards) · TruthfulQA | domain Elo | Helped |
| Increase general intelligence | Debaters (Houdinis) · TruthfulQA | domain Elo | No clear effect |
| Increase general intelligence | Models generating backdoors (Houdinis) · Backdoor Code | viable backdoor generation | Helped |
| Increase Houdini general intelligence, conditional on a viable backdoor | Backdoor Code · Backdoor Code | backdoor success against Guard detection | No clear effect |
| Modify the Wargames setup by limiting it to five actions and instructing the Narrator to be strict | Wargames · Wargames | Houdini Elo scaling | No clear effect |
Surprising findings
- Larger general Elo gap makes oversight easier in the theory, because Houdinis at the starting Guard's Elo get weaker under the bootstrapped protocol.
- Smarter Backdoor Code Houdinis produce more viable backdoors but are not more likely to evade the Guard once a viable backdoor exists.
Limitations
- Only single-step oversight simulated; real oversight may involve compounding dependencies between steps.
- Deceptive behavior is modeled via prompting rather than fine-tuning, since many models are proprietary.
- Chatbot Arena Elo is an imperfect proxy for general intelligence and the Wargames roleplay may not reflect real escape ability.
Experimental setup
Datasets: TruthfulQA BoolQ QuaLITY PrOntoQA APPS
Benchmarks: Mafia Debate Backdoor Code Wargames Counting-to-21
Metrics: Elo rating win rate log odds optimal number of oversight steps success probability
Baselines: Gemini-2.0-flash-001 GPT-4o Claude 3.5 Haiku
Builds on: Elo rating system Chatbot Arena AI safety via debate Akaike Information Criterion
Closest papers by meaning
- Calibrating Conservatism for Scalable Oversight (ICML 2026 · similarity 0.88)
- A theoretical case-study of Scalable Oversight in Hierarchical Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- The Scaling Laws of Skills in LLM Agent Systems (NeurIPS 2026 · similarity 0.84)
- CTRL-ALT-DECEIT Sabotage Evaluations for Automated AI R&D (NeurIPS 2025 · similarity 0.84)
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations (ICLR 2025 · similarity 0.83)
- Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.