Research map / Large language models / Language, safety and interpretability
Shared Lexical Task Representations Explain Behavioral Variability In LLMs
Zhuonan Yang, Xiaochen Li, Francisco Piedrahita Velez, Eric Todd, David Bau, Michael Littman, Stephen Bach, Ellie Pavlick
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| patch averaged lexical task head activations (scaled up) into zero-shot prompts (ablation) | lexical task heads · 17 simple tasks, Llama-3.1-8B-Instruct | task accuracy: recovers at least 10% to 90% of task performance across prompts | Improved |
| patch activations from a related control task instead of the target task (ablation) | lexical task heads · zero-shot prompts, Llama-3.1-8B-Instruct | task accuracy | No clear effect |
| scale up average lexical task head activation from correct prompts in incorrect prompts (ablation) | lexical task heads · incorrect prompts (baseline accuracy 0) | accuracy | Improved |
| strengthen lexical task head output (ablation) | retrieval heads · country-capital style tasks | retrieval heads retrieving correct answer | Improved |
| increase number of in-context shots | example-based prompts · 17 tasks | lexical task head count and output norm: 14 out of 17 tasks positively correlated with accuracy | Improved |
| add misleading ambiguous demonstration pair to example prompt | example-based prompts · Product-Producer task | target task head activation | Hurt |
| activate lexical task heads of both composing tasks (ablation) | two-hop compositional tasks · compositional tasks | performance recovered: 12%-50% | Improved |
| activate Python lexical task heads with a too strong scaling factor (ablation) | code steering · HumanEval-X JavaScript prompts | generation quality | Hurt |
Closest papers by meaning
- Do different prompting methods yield a common task representation in language models? (NeurIPS 2025 · similarity 0.91)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.89)
- Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (ICML 2025 · similarity 0.89)
- Do LLMs ``know'' internally when they follow instructions? (ICLR 2025 · similarity 0.88)
- Decomposing how prompting steers behavior (NeurIPS 2026 · similarity 0.88)
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
