Research map / Large language models / Language, safety and interpretability
The Achilles’ Heel of LLMs: How Altering a Handful of Neurons Can Cripple Language Abilities
Zixuan Qin, Qingchen Yu, Kunlin Lyu, Zhaoxin Fan, Yifan Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Mask 3 to 45 top-ranked critical neurons found by perturbation-based identification (ablation) | 21 LLMs 0.5B-72B · WikiText-103 | perplexity: 9.98 -> 6.25×10^21 (Gemma-7B, 3 neurons) | Hurt |
| Mask critical neurons in 70B models (ablation) | Llama-3.3-70B, DeepSeek-R1-Distill-Llama-70B, Qwen2.5-72B · seven downstream benchmarks | accuracy: e.g. MMLU-Pro 0.4441 -> 0.0000 | Hurt |
| Mask all four critical neurons jointly rather than any subset (ablation) | Llama-3.2-1B-Instruct · WikiText-103 | perplexity: subsets 18.1-24.9 vs full set 1585570.7467 | Hurt |
| Rank neurons by activation magnitude or gradient magnitude, or random, instead of perturbation-based importance (ablation) | neuron localization · 3 70B models, WikiText-103 | perplexity after masking up to 1000 neurons | Hurt |
| Mask up_proj/gate_proj neurons instead of down_proj (ablation) | Qwen2.5-0.5B, Llama-3.2-1B/3B · WikiText-103 | perplexity: 13.4696 -> 497.5569 (3B, top-1000) | Hurt |
| Change numerical precision fp16 to fp32/fp64 (ablation) | critical neuron masking · WikiText-103 | masked perplexity: 1.59×10^6 across fp16/fp32/fp64 (Llama-3.2-1B) | No clear effect |
| Raise noise scale alpha to >= 5 (ablation) | critical neuron identification · Llama-3.3-70B-Instruct | number of identified critical neurons: stabilizes at 7 neurons for alpha >= 5 | Improved |
| Raise sample size K above 80 (ablation) | critical neuron identification · Llama-3.3-70B-Instruct | number of identified critical neurons: stable at 7 neurons once K exceeds 80 | Improved |
| Raise criticality threshold epsilon from 1 to 2 or higher (ablation) | greedy search · Qwen2.5-32B-Instruct | neurons found: 45 -> 1000+ (convergence failure) | Hurt |
Closest papers by meaning
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.89)
- The Emergence of Abstract Thought in Large Language Models Beyond Any Language (NeurIPS 2025 · similarity 0.89)
- Do LLMs Signal When They’re Right? Evidence from Neuron Agreement (ICML 2026 · similarity 0.88)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.88)
- PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding (NeurIPS 2025 · similarity 0.88)
- Sparsing Law: Towards Large Language Models with Greater Activation Sparsity (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
