Research map / Large language models / Language, safety and interpretability
Control Illusion: The Failure of Instruction Hierarchies in Large Language Models
Yilin Geng, Haonan Li, Honglin Mu, Xudong Han, Timothy Baldwin, Omri Abend, Eduard Hovy, Lea Frermann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| place primary constraint in system message vs no priority designation (system/user separation) | six LLMs · Simple and rich conflicting constraints | primary obedience rate R1 (%): IF baseline 74.8-90.8 vs average R1 9.6-45.8 | No clear effect |
| add explicit priority declaration to system message (Emphasized) (ablation) | GPT4o · Simple instructions | primary obedience rate R1 (%): 47.0 -> 63.8 | Improved |
| add explicit priority declaration to system message (Emphasized) (ablation) | GPT4o-mini · Rich instructions | primary obedience rate R1 (%): 41.8 -> 43.6 | No clear effect |
| repeat task description in both system and user messages (Task) (ablation) | GPT4o · Simple instructions | primary obedience rate R1 (%): 47.0 -> 31.3 | Hurt |
| scale Llama from 8B to 70B | Llama-3.1 · Conflicting constraints | average primary obedience rate (%): 10.1 -> 16.4 | No clear effect |
| use larger GPT4o instead of GPT4o-mini | GPT-4o family · Conflicting constraints | average primary obedience rate (%): 45.8 -> 40.8 | Hurt |
| enrich prompts with richer context (ablation) | six LLMs · Conflicting constraints | primary obedience rate R1 (%) | No clear effect |
| frame dominant constraint with social consensus instead of system/user roles | GPT4o-mini · Priority adherence rate | PAR (%): 47.5 -> 77.8 | Improved |
| frame dominant constraint with social consensus instead of system/user roles | Qwen-7B · Priority adherence rate | PAR (%): 14.4 -> 65.8 | Improved |
| authority framing (CEO vs Intern) instead of system/user roles | Claude3.5-S · Priority adherence rate | PAR (%): 23.6 -> 32.4 | Improved |
| swap constraint ordering in consensus framing (ablation) | Claude3.5-S · Priority adherence rate | PAR (%): 85.8 -> 38.2 | Hurt |
| swap constraint ordering in authority/expertise framing (ablation) | Qwen · Priority adherence rate | PAR (%): 54.2 -> 53.9 (authority) | No clear effect |
Closest papers by meaning
- A Closer Look at the Limitations of Instruction Tuning (ICML 2024 · similarity 0.88)
- Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.88)
- DINGO: Towards Diverse and Fine-Grained Instruction-Following Evaluation (AAAI 2024 · similarity 0.88)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.87)
- Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · similarity 0.87)
- Can Large Language Models Understand Real-World Complex Instructions? (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
