Research map / Large language models / Language, safety and interpretability
Linguistic encoding in LLM representations
Probe how LLM embeddings encode syntax, semantics, and brain-aligned information.
123 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Language, safety and interpretability, keyword cluster “languages · pretraining · multilingual”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
24 papers in 2023–24 and 73 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.4% to 4.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Convolutional Neural Networks over Tree Structures for Programming Language Processing (AAAI 2016 · 536 citations)
- 2017 · Semantic Parsing with Neural Hybrid Trees (AAAI 2017 · 16 citations)
- 2018 · SenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings (AAAI 2018 · 415 citations)
- 2019 · Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) (NeurIPS 2019 · 140 citations)
- 2020 · Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech (NeurIPS 2020 · 30 citations)
- 2022 · Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps (ICML 2022 · 15 citations)
- 2023 · Scaling laws for language encoding models in fMRI (NeurIPS 2023 · 29 citations)
- 2024 · Language Models Represent Space and Time (ICLR 2024 · 42 citations)
- 2025 · SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking (ICLR 2025 · 4 citations)
- 2026 · Do Large Language Models Think like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI (AAAI 2026 · 1 citation)
Earliest papers here
- Convolutional Neural Networks over Tree Structures for Programming Language Processing (AAAI 2016 · 536 citations)
- Verb Pattern: A Probabilistic Semantic Representation on Verbs (AAAI 2016 · 6 citations)
- Semantic Parsing with Neural Hybrid Trees (AAAI 2017 · 16 citations)
What has been tried
80 papers here report 731 measured changes (a component added, removed or swapped, compared with a baseline): helped 60%, no clear effect 12%, hurt 21%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace the linear probe with a one-layer MLP with 256 neurons | World, USA, NYC, Historical, Entertainment, and Headlines · R² | World: 0.881 -> 0.897; USA: 0.799 -> 0.819; NYC: 0.219 -> 0.204; Historical: 0.785 -> 0.775; Entertainment: 0.788 -> 0.7 | Mixed | Language Models Represent Space and Time (ICLR 2024 · 42 citations) |
| Use two-stage training: first train the adapter with other parameters frozen, then fine-tune the full model | MovieLens 25M · convergence | Helped | Demystifying Embedding Spaces using Large Language Models (ICLR 2024 · 5 citations) | |
| unstructured per-element spiking steps in weights instead of structured channel-wise steps | common sense reasoning avg · avg accuracy | 48.90 to 56.11 | Helped | SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking (ICLR 2025 · 4 citations) |
| Reconstruct the function vector from only the top 100 decoded tokens instead of the original vector | Zero-shot; six tasks · accuracy | Antonym 48.2 ± 2.0 % -> 4.8 ± 2.0%; Capitalize 70.5 ± 2.4% -> 5.7 ± 2.2%; Country-Capital 83.2 ± 2.7% -> 58.1 ± 18.5%; E | Hurt | Function Vectors in Large Language Models (ICLR 2024 · 4 citations) |
| Use a 7B model instead of a 70B model | TextWorld · Interpretation responses not following instruction | 3.84% -> 27.17% | Hurt | SelfIE: Self-Interpretation of Large Language Model Embeddings (ICML 2024 · 4 citations) |
| collect activations using the Pile rather than RPJ-v2 | SAE feature interpretation evaluation · Detection | 0.74 (0.63--0.85) -> 0.76 (0.67--0.85) | No clear effect | Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · 3 citations) |
| Remove FFN updates carrying full-word representations | Suffix-split words · retrieval rate | 85% -> 18% | Hurt | From Tokens to Words: On the Inner Lexicon of LLMs (ICLR 2025 · 3 citations) |
| Replace matched fMRI targets with block-permuted fMRI targets | Late language regions · normalized brain alignment | Hurt | Improving Semantic Understanding in Speech Language Models via Brain-tuning (ICLR 2025 · 3 citations) | |
| increase the number of LLM-generated training examples | STS tasks (Avg.) · Spearman's correlation | Helped | Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · 3 citations) | |
| Replace the multi-token identity demonstrations with a position-matched single-token prompt | Next-token prediction estimation · Precision@1 | Hurt | Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models (ICML 2024 · 2 citations) | |
| Use spline features from the first 3 layers with a random forest | Omni-Toxic · ROC-AUC | 94.68 | Helped | Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation (ICML 2024 · 2 citations) |
| clip neuron 11.2378 activation to its mean from the first occurrence | Synthetic induction (duplicated 200-token C4 sequences) · entropy | up to a 70\% reduction in entropy | Hurt | Confidence Regulation Neurons in Language Models (NeurIPS 2024 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 41 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- The Pile · 4 of 41 papers
- MMLU · 4 of 41 papers
- C4 · 3 of 41 papers
- WikiText · 3 of 41 papers
- AG News · 3 of 41 papers
- Dolma · 2 of 41 papers
- PIQA · 2 of 41 papers
- SST2 · 2 of 41 papers
Baselines
- ReLU SAE · 2 of 41 papers
- TopK SAE · 2 of 41 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 97 papers; from OpenAlex and the papers' affiliations).
- Massachusetts Institute of Technology · 8 papers
- Google · 6 papers
- Google DeepMind · 5 papers
- Northeastern University · 4 papers
- Zhejiang University · 3 papers
- Columbia University · 3 papers
- University of California, Berkeley · 3 papers
- Meta · 3 papers
- Tsinghua University · 2 papers
- University of Toronto · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Pruning and circuits in LLMs (135 papers)
- Reasoning mechanisms and failure modes (114 papers)
- Task vectors and steering (84 papers)
Elsewhere in the topic
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Uncertainty quantification for LLMs (136 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).