Research map / Large language models / Language, safety and interpretability
Operationalising the Superficial Alignment Hypothesis via Task Complexity
Tomás Vergara Browne, Darshan Patil, Ivan Titov, Siva Reddy, Tiago Pimentel, Marius Mosbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt pre-trained model with a short program (a few kilobits) | Olmo3-32B · GSM8K | accuracy (%): 29.6 -> 72.2 | Improved |
| adapt pre-trained model with a short program | Olmo3-7B · FLORES English-French | BLEU: 22.63 -> 34.43 | Improved |
| enlarge program from ImageNet-image size to the largest programs (ablation) | Olmo3-32B · GSM8K | accuracy (%): 78.4% | No clear effect |
| add pre-training versus random initialisation (ablation) | Olmo3-7B · GSM8K | accuracy (%): 1.1% -> 67.6% | Improved |
| add pre-training versus random initialisation (ablation) | Olmo3-7B · IFEval | score (%): 26.6% -> 68.5% | Improved |
| add post-training on top of pre-trained checkpoint (ablation) | SmolLM3 3B · GSM8K | accuracy (%): 58.4% -> 82.8% | Improved |
| add post-training on top of pre-trained checkpoint (ablation) | SmolLM3 3B · IFEval | score (%): 49.6% -> 70.5% | Improved |
| add post-training to reduce program length needed for max performance (ablation) | Olmo3-7B · GSM8K | program length (bits): 5 × 10^6 bits -> minimal effect beyond 10^4 bits | Improved |
| use output-layer linear projection fine-tuning instead of Pareto methods (ablation) | Chen et al. linear projection · GSM8K (Olmo3-7B) | accuracy vs program length | Hurt |
| increase program length from 10^5 bits for pre-trained checkpoints (ablation) | pre-trained checkpoints · GSM8K and IFEval | accuracy | Improved |
Closest papers by meaning
- Training on the Test Task Confounds Evaluation and Emergence (ICLR 2025 · similarity 0.90)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.90)
- Quantifying Generalization Complexity for Large Language Models (ICLR 2025 · similarity 0.89)
- EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving (NeurIPS 2025 · similarity 0.89)
- An Analysis for Reasoning Bias of Language Models with Small Initialization (ICML 2025 · similarity 0.88)
- Predicting Emergent Abilities with Infinite Resolution Evaluation (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
