Research map / Large language models / AI and society
The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?
Alexander Hägele, Aryo Pradipta Gema, Henry Sleight, Ethan Perez, Jascha Sohl-Dickstein
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| longer reasoning / more actions per question | Sonnet 4, o3-mini, o4-mini, Qwen3 · GPQA, MMLU, SWE-Bench Verified, Model-Written Evals | error-incoherence | Hurt |
| split samples by above-median reasoning length for the same question (ablation) | frontier models · GPQA and SWE-Bench Verified | error-incoherence | Hurt |
| increase Qwen3 model size on easy questions (ablation) | Qwen3 (up to 32B) · MMLU, easiest question group | error-incoherence | Improved |
| increase Qwen3 model size on hardest questions (ablation) | Qwen3 (up to 32B) · MMLU, hardest question group | error-incoherence | Hurt |
| increase Qwen3 model size (ablation) | Qwen3 (up to 32B) · MMLU / GPQA | accuracy | Improved |
| increase transformer size in synthetic optimizer task (ablation) | transformers (47K to 5M params) on quadratic descent · synthetic optimizer | error-incoherence | Hurt |
| increase transformer size in synthetic optimizer task (ablation) | transformers (47K to 5M params) on quadratic descent · synthetic optimizer | next-token loss | Improved |
| larger instructed reasoning budget (ablation) | o3-mini, o4-mini, Qwen3 (not Sonnet 4) · GPQA | error-incoherence | Improved |
| larger instructed reasoning budget (ablation) | frontier models · GPQA | accuracy | Improved |
| larger instructed reasoning budget (ablation) | Sonnet 4 · GPQA | error-incoherence | No clear effect |
| ensemble multiple samples (averaged probabilities) (ablation) | o4-mini · GPQA, ensemble sizes up to 32 | variance | Improved |
| ensemble multiple samples (averaged probabilities) (ablation) | o4-mini · GPQA, ensemble sizes up to 32 | error-incoherence | Improved |
Closest papers by meaning
- Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors (NeurIPS 2026 · similarity 0.88)
- Towards a Science of AI Agent Reliability (ICML 2026 · similarity 0.87)
- Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment (AAAI 2025 · similarity 0.87)
- Measuring Error Alignment for Decision-Making Systems (AAAI 2025 · similarity 0.86)
- The Generative AI Paradox: “What It Can Create, It May Not Understand” (ICLR 2024 · similarity 0.86)
- Great Models Think Alike and this Undermines AI Oversight (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
