Research map / Large language models / Language, safety and interpretability

Scaling and context steer LLMs along the same computational path as the human brain

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Compares the temporal order of LLM layer representations with the order of MEG brain responses during 10 hours of audiobook listening, across 22 LLMs, finding that early-to-deep layers align with early-to-late brain responses, increasingly with model size and context length.

Problem

Whether LLM-brain representational alignment reflects a similar sequence of computations over time, and which factors (architecture, size, context length) drive this temporal alignment, remained unknown.

Approach

Introduces a "temporal score" measuring whether the time at which each LLM layer best aligns with MEG responses correlates with layer depth, and uses it to show a shared computational order between LLMs and the brain that emerges with model size and context length.

Key results

• Temporal score averaged across nine LLMs is r = 0.99 (p < 1e-06) between layer depth and MEG peak latency.
• In the Pythia family, temporal score rises from non-significant (r = 0.44) at 14M parameters to r = 0.96 (p < 1e-4) at 12B.
• For Llama-3.2 3B, temporal score rises from non-significant r = 0.19 without context to r = 0.93 at 1000-word context.

Measured effects reported in the paper

Surprising findings

• Bidirectional models (BERT, RoBERTa, Wav2vec2.0) achieve comparable alignment scores but substantially lower, non-significant temporal scores than causal LLMs.
• Word predictability does not explain temporal alignment: temporal alignment holds for both most expected and most surprising word quartiles.

Limitations

• MEG has limited spatial resolution and cannot capture deep sources or single neurons.
• Only three subjects in the dataset, so inter-individual variability is unknown.
• Only text models are studied; no sensory (speech) modality input.

Experimental setup

Datasets: public MEG dataset of 3 participants listening to 10 h of audiobooks

Benchmarks: MEG responses to 10 h audiobook listening

Metrics: Pearson correlation alignment score temporal score (correlation between peak time and layer dept

Baselines: BERT RoBERTa Wav2vec2.0

Builds on: Pythia ridge regression encoding models

Closest papers by meaning

• Divergences between Language Models and Human Brains (NeurIPS 2024 · similarity 0.90)
• Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models (NeurIPS 2025 · similarity 0.89)
• Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain) (ICLR 2025 · similarity 0.88)
• PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding (NeurIPS 2025 · similarity 0.88)
• Improving Semantic Understanding in Speech Language Models via Brain-tuning (ICLR 2025 · similarity 0.88)
• fMRI predictors based on language models of increasing complexity recover brain left lateralization (NeurIPS 2024 · similarity 0.88)