Research map / Large language models / Language, safety and interpretability
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace learned attention with uniform attention in shallow BERT layers (layers 1-6) (ablation) | BERT · 13 subject-verb agreement tasks | accuracy | Helped |
| Replace learned attention with uniform attention in layer 11 (ablation) | BERT · 13 subject-verb agreement tasks | accuracy | Hurt |
| Replace learned attention with uniform attention in shallow layers (ablation) | BERT · brain activity prediction | prediction accuracy | Helped |
| Replace learned attention with uniform attention in deep layers (ablation) | BERT · brain activity prediction | prediction accuracy | Hurt |
| Increase context length (ablation) | Transformer-XL · Harry Potter fMRI brain activity prediction | prediction accuracy | Helped |
| Increase context length (ablation) | ELMo and BERT · Harry Potter fMRI brain activity prediction | prediction accuracy | Mixed |
| Use middle rather than other layers for contexts longer than 15 words (ablation) | ELMo, BERT, and Transformer-XL · Harry Potter fMRI brain activity prediction | prediction accuracy | Helped |
| Use deep-layer representations at fewer than 10 words of context (ablation) | ELMo, BERT, and Transformer-XL · Harry Potter fMRI brain activity prediction | prediction accuracy | Mixed |
| Use a long-context sentence representation rather than a current-word embedding (ablation) | USE · Harry Potter fMRI brain activity prediction | prediction accuracy | Mixed |
Closest papers by meaning
- Inducing brain-relevant bias in natural language processing models (NeurIPS 2019 · similarity 0.92)
- Improving Semantic Understanding in Speech Language Models via Brain-tuning (ICLR 2025 · similarity 0.90)
- Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement (NeurIPS 2025 · similarity 0.89)
- Divergences between Language Models and Human Brains (NeurIPS 2024 · similarity 0.87)
- The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities (ICLR 2025 · similarity 0.87)
- Scaling and context steer LLMs along the same computational path as the human brain (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.