Research map / Large language models / Language, safety and interpretability
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies language-related neurons in 20 open LLMs, splits them into shared and exclusive sets, shows shared neurons grow in proportion and importance with model generation, and uses this to pick which neuron group to continually pretrain for multilingual gains.
Problem
Whether LLMs reason in a language-specific space or a higher-level language-agnostic space, and whether such an abstract space emerges as models develop.
Approach
Defines language-shared vs. language-exclusive neurons via ablation-based activation, and a Language Agnostic Score comparing perplexity impact of shared vs. exclusive neurons; the score guides which neuron group to fine-tune (all language-related for low scores, shared for mid, exclusive for high).
Key results
- Shared neuron ratio correlates with multilingual ability within model series (average Pearson R = 0.92, Spearman rho = 0.88).
- In Qwen2.5-series models, deactivating shared neurons raises perplexity sharply, often orders of magnitude more than removing exclusive neurons, while early models (Qwen1.5, LLaMA-1-7B) show Language Agnostic Scores near 1.
- Training shared neurons on Llama-3.2-3B gives +5.7 average MGSM gain; training exclusive neurons on Llama-3.1-8B gives +4.0 MGSM gain, with only 100k general documents per language.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train language-exclusive neurons | Llama-3.1-8B · MGSM | average score change (ΔAvg): +4.0 | Helped |
| Train language-shared neurons | Llama-3.1-8B · MGSM | average score change (ΔAvg): 0.3 | No clear effect |
| Train language-exclusive neurons | Llama-3.1-8B · MMMLU | average score change (ΔAvg): +1.7 | Helped |
| Train language-shared neurons | Llama-3.2-3B · MGSM | average score change (ΔAvg): +5.7 | Helped |
| Train language-exclusive neurons | Llama-3.2-3B · MGSM | average score change (ΔAvg): +3.6 | Helped |
| Train language-shared neurons | Llama-3.2-3B · MMMLU | average score change (ΔAvg): 0.4 | No clear effect |
| Train language-shared neurons | Llama-3.2-1B · MGSM | average score change (ΔAvg): +3.1 | Helped |
| Train language-exclusive neurons | Llama-3.2-1B · MGSM | average score change (ΔAvg): +4.2 | Helped |
| Train language-exclusive neurons | Llama-3.2-1B · MMMLU | average score change (ΔAvg): 0.1 | No clear effect |
Surprising findings
- Early-stage models show shared neurons with importance comparable to exclusive neurons, i.e., shared overlap is superficial there.
- Gains on MMMLU are much smaller than on MGSM, suggesting the training mainly improves reasoning rather than factual recall.
Limitations
- Evaluation limited to smaller LLMs (up to about 9B) due to computational resources.
- Language-agnostic interpretation rests on perplexity-based ablation importance rather than direct evidence of abstract reasoning.
- Only six languages used for neuron analysis, with MGSM and MMMLU gains of a few points.
Experimental setup
Datasets: OSCAR Culturax MADLAD Wikipedia MMMLU MGSM GSM8K MMLU
Benchmarks: MMMLU MGSM
Metrics: accuracy perplexity change (delta PPL) Pearson correlation Spearman correlation Language Agnostic Score Shared Neuron Ratio
Baselines: Random neuron deactivation Shared-neuron training Exclusive-neuron training Random-neuron training
Builds on: parallel neuron detection (Bricken-style ablation neuron identification)
Closest papers by meaning
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.91)
- Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models (ICML 2025 · similarity 0.89)
- The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities (ICLR 2025 · similarity 0.89)
- An Analysis for Reasoning Bias of Language Models with Small Initialization (ICML 2025 · similarity 0.88)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.88)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.