Research map / Large language models / Language, safety and interpretability

Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models

ICML 2025 · Large language models › Language, safety and interpretability · approach: Reasoning mechanisms and failure modes · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper identifies an emergent three-stage symbolic architecture (symbol abstraction heads, symbolic induction heads, retrieval heads) in open-source LLMs that supports abstract rule induction, validated with causal mediation, attention, RSA, and ablation analyses.

Problem

Whether LLMs' emergent abstract reasoning relies on structured, symbol-processing mechanisms or merely statistical approximation of training data.

Approach

Proposes that abstract reasoning in LLMs is implemented by three sequential attention-head mechanisms: early-layer heads convert tokens into invariant abstract variables based on relations, intermediate-layer heads perform sequence induction over those variables (overlapping with function-vector heads), and late-layer retrieval heads map predicted variables back to tokens via indirection.

Key results

• Llama-3.1 70B achieves 95% 2-shot accuracy on the algebraic rule induction task, and 99.2% (successor) and 82.0% (predecessor) 2-shot accuracy on letter string analogies.
• Symbolic induction head causal mediation scores correlate highly with function vector scores at the final position (r=0.86) but weakly with standard induction head prefix matching scores (r=0.11).
• Ablating the top-ranked symbol abstraction, symbolic induction, or retrieval heads rapidly drives the correct-answer probability to zero, while control ablations require nearly all heads to produce a similar effect; GPT-2 models lack robust symbol abstraction heads.

Measured effects reported in the paper

Surprising findings

• Symbolic induction head scores are only weakly correlated with standard induction head scores (r=0.11), suggesting disjoint head sets despite conceptual similarity.
• Verbal analogy task causal mediation correlations within and between tasks are lower or even negative, indicating specialization across tasks.

Limitations

• Analyses focus on simple synthetic tasks (identity rules) and extend to only two other tasks.
• Representations of symbol abstraction and induction heads still carry token identity, so the mechanisms do not exclusively represent abstract variables.
• GPT-2 results are equivocal and the claim that symbolic mechanisms emerge only at certain scales is not fully established.

Experimental setup

Datasets: synthetic algebraic identity rule prompts (ABA/ABB) with random tokens letter string analogy prompts (successor/predecessor) verbal analogy prompts (antonym/synonym)

Benchmarks: identity rules task (ABA/ABB) letter string analogies verbal analogies

Metrics: accuracy causal mediation score probability of correct answer under ablation Pearson correlation for RSA decoding generalization accuracy prefix matching score

Baselines: GPT-2 variants random head ablation control head ablation

Builds on: induction heads function vectors abstractor architecture causal mediation analysis

Closest papers by meaning

• Benchmarking Abstract and Reasoning Abilities Through A Theoretical Perspective (ICML 2025 · similarity 0.90)
• The Emergence of Abstract Thought in Large Language Models Beyond Any Language (NeurIPS 2025 · similarity 0.89)
• NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models (AAAI 2026 · similarity 0.89)
• Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective (ICML 2026 · similarity 0.88)
• How do Transformers Learn Implicit Reasoning? (NeurIPS 2025 · similarity 0.87)
• Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought (ICLR 2026 · similarity 0.87)