Research map / Large language models / Language, safety and interpretability
Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a residual disentanglement method that isolates lexicon, syntax, meaning, and reasoning embeddings from Qwen2.5-14B layers and uses them to model ECoG responses to natural speech, finding a late, extra-linguistic reasoning signal.
Problem
Standard LLM embeddings entangle lexical, syntactic, semantic and reasoning information, so brain encoding models are biased toward shallow features and cannot isolate neural substrates of reasoning during language comprehension.
Approach
Probe layers to find saturation points for syntax, meaning and reasoning, then iteratively regress lower-level features out of higher-layer hidden states via ridge regression to obtain near-orthogonal residual embeddings used as brain encoding features.
Key results
- Saturation layers for Qwen2.5-14B are 6 (syntax), 20 (meaning), and 30 (reasoning); residual embeddings have off-diagonal mean absolute cosine similarity of at most 0.045.
- Reasoning-selective electrodes peak at about 362 ms after word onset, later than lexicon, syntax and meaning signals.
- Shallow features (lexicon, syntax) yield significantly higher peak correlations than meaning and reasoning embeddings (Welch's t-test, p<0.001), while reasoning embeddings show significantly higher correlations in visual cortex electrodes than other features.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use residual feature embeddings instead of undisentangled full LLM embeddings | Brain encoding of Podcast ECoG responses · Podcast ECoG | unique neural variance explained by reasoning features | Helped |
| use reasoning residual embeddings rather than lexical, syntactic, or meaning features | Brain encoding of Podcast ECoG responses · Podcast ECoG | peak latency: ~362 ms | Mixed |
| disentangle linguistic features rather than use raw hidden states | Qwen2.5-14B representations · Token-level feature similarity | mean absolute cosine similarity: 0.751 -> all <= 0.045 | Helped |
| evaluate syntax residual on the meaning probe rather than its target syntax task (ablation) | Qwen2.5-14B syntax residual · COMPS-BASE | accuracy relative to BoW baseline: 0.665 vs. 0.589 | Hurt |
| evaluate reasoning residual on the meaning probe rather than its target reasoning task (ablation) | Qwen2.5-14B reasoning residual · COMPS-BASE | accuracy relative to BoW baseline: 0.665 vs. 0.605 | Hurt |
| increase model size and use newer Qwen generations | Qwen family · ProntoQA, COMPS-WUGS-DIST, and WinoGrande | accuracy | Mixed |
| use the full, non-disentangled embedding instead of separated feature representations | Brain encoding of Podcast ECoG responses · Podcast ECoG | isolation of reasoning-related temporal activity | Hurt |
Surprising findings
- Lexicon embedding activates the fewest electrodes despite having high peak correlations.
- Reasoning-selective electrodes extend into visual cortex, beyond classical language areas.
Limitations
- ECoG coverage is sparse in frontal regions, which are crucial for reasoning.
- Only one 30-minute podcast episode and nine participants; correlational encoding results.
- Residual orthogonality is approximate and relies on a linear-projection assumption and probing-based layer selection.
Experimental setup
Datasets: BLiMP COMPS-BASE COMPS-WUGS-DIST ProntoQA WinoGrande Podcast ECoG dataset expanded podcast transcripts (16 episodes)
Benchmarks: BLiMP COMPS-BASE COMPS-WUGS-DIST ProntoQA WinoGrande Podcast ECoG dataset
Metrics: normalized accuracy Pearson correlation z-score vs. shuffle null cosine similarity
Baselines: full LLM embedding lexicon embedding syntax embedding meaning embedding
Builds on: minimal pair probing ridge regression encoding models
Closest papers by meaning
- Probing Brain Activation Patterns by Dissociating Semantics and Syntax in Sentences (AAAI 2020 · similarity 0.91)
- When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (NeurIPS 2025 · similarity 0.89)
- Improving Semantic Understanding in Speech Language Models via Brain-tuning (ICLR 2025 · similarity 0.88)
- Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions (NeurIPS 2024 · similarity 0.88)
- Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models (NeurIPS 2025 · similarity 0.87)
- The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.