Research map / Large language models / Retrieval and knowledge
How do Language Models Bind Entities in Context?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies a binding ID mechanism in sufficiently large Pythia and LLaMA models, showing via causal mediation that entity-attribute bindings are represented by additive, geometrically structured binding ID vectors.
Problem
How language models bind entities to their attributes in context (the binding problem), so that they can later recall the correct attribute for each entity.
Approach
LMs attach abstract binding ID vectors to entity and attribute activations by vector addition; queries retrieve the attribute sharing the entity's binding ID. Causal mediation experiments (factorizability, position independence, additive mean interventions) verify this, and binding IDs form a continuous subspace where distance reflects discernability.
Key results
- Mean interventions with binding ID difference vectors flip beliefs: accuracy above 99% for control and below 3% for attribute and entity interventions on LLaMA-30b capitals.
- Binding ID difference vectors transfer across tasks: mean accuracy 0.87-0.88 on capitals for parallel and capitals sources versus 0.30-0.31 for zero and random baselines.
- Intervention effectiveness increases with model scale, with binding IDs present in every sufficiently large Pythia and LLaMA model studied.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Swap the attribute binding ID vectors | LLaMA 30B binding ID mechanism · Capitals, n=2 | median-calibrated accuracy: 0.99 -> 0.00; 1.00 -> 0.03 | Hurt |
| Swap the entity binding ID vectors | LLaMA 30B binding ID mechanism · Capitals, n=2 | median-calibrated accuracy: 0.99 -> 0.00; 1.00 -> 0.01 | Hurt |
| Apply entity and attribute mean interventions together | LLaMA 30B binding ID mechanism · Capitals, n=2 | median-calibrated accuracy: 0.99 -> 0.97; 1.00 -> 0.99 | No clear effect |
| Replace estimated binding ID difference vectors with random directions | LLaMA 30B binding ID mechanism · Capitals, n=2 | median-calibrated accuracy: 0.99 -> 0.98; 1.00 -> 0.98 | No clear effect |
| Apply binding-vector differences estimated from parallel | LLaMA 30B binding ID mechanism · Capitals, n=3 | mean median-calibrated accuracy: 0.30 -> 0.87 | Helped |
| Apply binding-vector differences estimated from shapes | LLaMA 30B binding ID mechanism · Capitals, n=3 | mean median-calibrated accuracy: 0.30 -> 0.71 | Helped |
| Apply binding-vector differences estimated from fruits | LLaMA 30B binding ID mechanism · Capitals, n=3 | mean median-calibrated accuracy: 0.30 -> 0.80 | Helped |
| Apply binding-vector differences estimated from bios | LLaMA 30B binding ID mechanism · Capitals, n=3 | mean median-calibrated accuracy: 0.30 -> 0.47 | Helped |
| Replace source-task binding vectors with randomly oriented vectors | LLaMA 30B binding ID mechanism · Capitals, n=3 | mean median-calibrated accuracy: 0.30 -> 0.31 | No clear effect |
| Swap the attribute binding ID vectors | LLaMA 65B binding ID mechanism · Onehop | median-calibrated accuracy: 0.73 -> 0.25; 0.79 -> 0.28 | Hurt |
| Swap the entity binding ID vectors | LLaMA 65B binding ID mechanism · Onehop | median-calibrated accuracy: 0.73 -> 0.24; 0.79 -> 0.26 | Hurt |
| Apply entity and attribute mean interventions together | LLaMA 65B binding ID mechanism · Onehop | median-calibrated accuracy: 0.73 -> 0.71; 0.79 -> 0.77 | No clear effect |
Surprising findings
- In an MCQ option-label task, binding is direct rather than via binding IDs, showing binding IDs are not universal.
- Larger models show more structured binding representations, contrary to the intuition that scale makes models harder to interpret.
Limitations
- Binding ID mechanism is not universal; direct binding is used for MCQ tasks.
- Attributes spanning many tokens (bios) make attribute mean interventions ineffective.
- Position-dependent bias complicates position independence interpretation and requires calibration.
Experimental setup
Datasets: SST-2
Benchmarks: capitals parallel shapes fruits bios onehop SST-2 MCQ
Metrics: median-calibrated accuracy mean log prob top-1 accuracy
Baselines: direct binding (alternative mechanism) random vectors zero vector ablation
Builds on: rotary position embeddings causal mediation analysis median-calibrated accuracy
Closest papers by meaning
- Monitoring Latent World States in Language Models with Propositional Probes (ICLR 2025 · similarity 0.84)
- KScope: A Framework for Characterizing the Knowledge Status of Language Models (NeurIPS 2025 · similarity 0.84)
- On Linear Representations and Pretraining Data Frequency in Language Models (ICLR 2025 · similarity 0.84)
- Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.84)
- Taming Knowledge Conflicts in Language Models (ICML 2025 · similarity 0.84)
- What does the Knowledge Neuron Thesis Have to do with Knowledge? (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.