Research map / Large language models / Language, safety and interpretability
Artificial Aphasias in Lesioned Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Zero-ablates (Bernoulli-masks) attention and FFN weight matrices in five ~1B LMs and scores the outputs with the Text Aphasia Battery (Gemini 2.5 Flash scorer), comparing the resulting symptom profiles to AphasiaBank human productions.
Problem
Whether lesion-induced failures in language models reproduce human aphasia syndromes, and how LM component and layer damage maps onto clinically defined symptom profiles.
Approach
Applies the aphasia-inspired lesion paradigm at fine granularity: component-by-layer-by-severity lesions of Q/K/V/O/Gate/Up/Down matrices, scored in a shared text-only symptom space with human AphasiaBank productions, to test both similarities and differences with human aphasia.
Key results
- Attention and FFN lesions produce separable symptom profiles (all-symptom L2 3.73 pp, sign-flip p=0.0004 over 360 strata).
- Lesions in early layers disproportionately cause semantic and syntactic symptoms, while late-middle layers yield more phonological and fluency symptoms.
- Human AphasiaBank productions show higher semantic/syntactic symptom rates (e.g., Meaning unclear 45.2% vs 14.4% in LMs), while LMs overproduce repetition-loop symptoms (43.4% vs 10.9%).
Surprising findings
- Semantic and syntactic deficits localize to earlier layers than phonological and fluency deficits.
- Lesioned LM symptom profiles differ qualitatively from human aphasia profiles despite some coarse category-level convergence.
Limitations
- Dependence on Gemini as the symptom scorer; individual symptom directions are scorer-dependent.
- Zero/Bernoulli masking can create out-of-distribution model states; mean-replacement only partly mitigates this.
- Primary analyses use a 20-prompt TAB subset and greedy decoding, which can amplify repetition loops.
Experimental setup
Datasets: AphasiaBank
Benchmarks: Text Aphasia Battery
Metrics: symptom rate L2 profile distance cosine similarity co-occurrence phi Cohen's d
Baselines: AphasiaBank human productions
Builds on: Text Aphasia Battery Quick Aphasia Battery AphasiaBank
Closest papers by meaning
- Improving Semantic Understanding in Speech Language Models via Brain-tuning (ICLR 2025 · similarity 0.86)
- fMRI predictors based on language models of increasing complexity recover brain left lateralization (NeurIPS 2024 · similarity 0.85)
- TopoLM: brain-like spatio-functional organization in a topographic language model (ICLR 2025 · similarity 0.85)
- Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain (NeurIPS 2025 · similarity 0.85)
- Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models (NeurIPS 2025 · similarity 0.85)
- Divergences between Language Models and Human Brains (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.