Research map / Large language models / Language, safety and interpretability
Benchmarking and Understanding Compositional Relational Reasoning of LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Generalized Associative Recall (GAR), a synthetic benchmark unifying associative recall, knowledge recall and IOI for compositional relational reasoning, evaluates LLMs on it, and uses attribution patching to find circuits and True/False attention heads in Vicuna-33B.
Problem
Whether and how existing transformer LLMs solve compositional relational reasoning (CRR) tasks, given that prior synthetic benchmarks are either too hard for open LLMs to analyze mechanistically or too simple to expose LLM deficiencies, and MI studies focus on single tasks.
Approach
Unify associative recall, knowledge recall and IOI into one framework of relational loops, where non-"same" semantic relations and negation/classification variations control difficulty; then apply attribution patching to reverse-engineer circuits in a mid-sized LLM and validate identified heads via intervention.
Key results
- GPT-4 reaches only 71.5% average accuracy on GAR, and accuracy on the hardest (other,other + negate) generation tasks drops below 40%.
- Strong intervention on upstream heads raises Vicuna-33B accuracy, and True/False head interventions increase average accuracy of Vicuna-7B/13B/33B by 17%/14%/6%.
- An MLP on four True/False head activations of Vicuna-33B reaches 87.07% on SNLI and 85.63% on GoT subsets, approaching or surpassing Vicuna-7B one-shot accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model scale | LLMs evaluated on GAR · GAR | accuracy | Helped |
| Combine two non-same semantic relations and add negation | GAR generation tasks · GAR | accuracy | Hurt |
| Apply syntactic variations such as swapQA or swapKV | GAR tasks · GAR | accuracy | No clear effect |
| Strongly intervene on task-relevant attention heads | Vicuna-33B · GAR generation and classification tasks | accuracy | Helped |
| Weakly intervene on task-relevant attention heads | Vicuna-33B · GAR tasks | accuracy | Helped |
| Intervene on pre-predicting heads | Vicuna-33B generation circuits · GAR generation tasks | accuracy | Helped |
| Intervene on True/False heads while knocking out the opposite-class heads | Vicuna-7B, Vicuna-13B, and Vicuna-33B · GAR classification tasks | accuracy: 17%/\/14%/6% | Helped |
| Use True/False head activations as MLP features | Vicuna-33B activations · SNLI and GoT | accuracy: 87.07$\pm$0.2; 85.63$\pm$1.8 | Helped |
| Knock out True/False heads and intervene on the opposite-class heads (ablation) | Vicuna-33B · GAR classification examples sampled as correct | accuracy: 100% to 60% | Hurt |
| Prepend randomly shuffled candidate answers when r_retrieve is not same (ablation) | GAR prompts · GAR | accuracy | Helped |
| Bracket the K-V pairs with angle brackets (ablation) | GAR prompts · GAR | accuracy | Helped |
| Label K-V pairs as the premise and add an Answer with Yes or No instruction (ablation) | GAR classification prompts · GAR | accuracy | Helped |
Surprising findings
- Scaling improves GAR accuracy but the compositionality gap grows for the Llama series including Vicuna as models scale.
- Syntactic variations (swapKV, swapQA) have much less impact on performance than semantic variations.
Limitations
- Circuit discovery is manual and restricted to 8 tasks; automatic circuit discovery methods were not used.
- GAR is programmatically generated from templates and only two-hop, limiting realism.
- Mechanistic analysis is limited to Vicuna models (and a few others for validation).
Experimental setup
Datasets: SNLI Geometry of Truth (GoT)
Benchmarks: GAR SNLI Geometry of Truth
Metrics: accuracy average probability of correct answer compositionality gap attention weight matched to pattern intervention accuracy
Baselines: Llama-2-7B Llama-2-13B Llama-3-8B Llama-3-70B Yi-34B GPT-3.5-turbo GPT-4-turbo Vicuna-7B Vicuna-13B GPT-2-Medium GPT-2-XL
Builds on: associative recall indirect object identification induction heads path patching attribution patching
Closest papers by meaning
- Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning (ICLR 2026 · similarity 0.88)
- Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.87)
- Compositional Causal Reasoning Evaluation in Language Models (ICML 2025 · similarity 0.87)
- MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation (ICML 2025 · similarity 0.87)
- Operads for compositional reasoning in LLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.