Research map / Large language models / Retrieval and knowledge
The Rise of Parameter Specialization for Knowledge Storage in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes 20 open-source LLMs via masking MLP value vectors on a new Wikipedia-based benchmark (SpecWiki), showing that stronger models show higher parameter specialization for knowledge, and that fine-tuning highly activated vectors improves knowledge use and reduces hallucination.
Problem
How knowledge is stored in MLP parameters of LLMs and whether the degree of parameter specialization for knowledge relates to model capability and efficient knowledge utilization.
Approach
Defines a Parameter Specialization Score (PSS) measured by masking the top-k knowledge-activated MLP value vectors and comparing accuracy drops on concept-specific versus irrelevant questions; finds PSS correlates with performance across 20 LLMs, and that fine-tuning only the highly activated vectors (FT-PV) increases specialization and reduces hallucination.
Key results
- PSS correlates with SpecWiki MCQ performance across 20 models with Pearson 0.92 and Spearman 0.93.
- On Qwen2-7B, FT-PV raised MCQ accuracy from 0.72 to 0.77 and PSS from 0.124 to 0.133, while reducing semantic entropy from 0.56 to 0.39.
- Qwen1.5 PSS rises from 0.019 (0.5B) to 0.184 (14B) alongside MCQ accuracy rising from 0.61 to 0.82.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the top k/8 highly activated MLP parameter vectors using additional knowledge data | LLaMA2-7B · SpecWiki high-frequency concepts | MCQ accuracy: 0.60 -> 0.67 | Helped |
| Fine-tune the top k/8 highly activated MLP parameter vectors using additional knowledge data | Qwen2-7B · SpecWiki high-frequency concepts | OEG accuracy: 0.63 -> 0.70 | Helped |
| Fine-tune all MLP parameter vectors instead of the targeted subset (ablation) | LLaMA2-7B · SpecWiki high-frequency concepts | MCQ accuracy: 0.60 -> 0.63 | Helped |
| Fine-tune complementary MLP parameter vectors, excluding target vectors (ablation) | LLaMA2-7B · SpecWiki high-frequency concepts | OEG accuracy: 0.51 -> 0.51 | No clear effect |
| Fine-tune a random subset of MLP parameter vectors (ablation) | LLaMA2-7B · SpecWiki high-frequency concepts | MCQ accuracy: 0.60 -> 0.58 | Hurt |
| Fine-tune all MLP parameter vectors instead of the targeted subset (ablation) | Qwen2-7B · SpecWiki high-frequency concepts | OEG accuracy: 0.63 -> 0.67 | Helped |
| Fine-tune complementary MLP parameter vectors, excluding target vectors (ablation) | Qwen2-7B · SpecWiki high-frequency concepts | OEG accuracy: 0.63 -> 0.65 | Helped |
| Fine-tune a random subset of MLP parameter vectors (ablation) | Qwen2-7B · SpecWiki high-frequency concepts | OEG accuracy: 0.63 -> 0.63 | No clear effect |
| Increase model size within the Qwen1.5 series | Qwen1.5 · SpecWiki MCQ | PSS: 0.019 -> 0.184 | Helped |
| Increase model size within the Qwen1.5 series | Qwen1.5 · SpecWiki MCQ | accuracy: 0.61 -> 0.82 | Helped |
| Mask a small proportion (5% to 10%) of concept-related vectors | Models in the LLaMA, Qwen, Mistral, and Gemma families · Unrelated questions | accuracy | Helped |
| Fine-tune only the top k/8 highly activated vectors rather than k/2, k/4, or k/16 (ablation) | LLaMA2-7B and Qwen2-7B · SpecWiki knowledge tasks | performance | Helped |
Surprising findings
- Masking a small proportion (5-10%) of concept-related value vectors made models perform better on unrelated questions than the original models.
- Parameter specialization emerges only in later OLMo-2 pretraining checkpoints, not in early training.
Limitations
- Only MLP value vectors analyzed; attention-based knowledge storage not studied.
- Experiments limited to models up to 14B parameters due to GPU limits.
- Causal experiments use only 10 concepts and two base models.
Experimental setup
Datasets: SpecWiki The Pile Wikipedia page views
Benchmarks: SpecWiki MMLU
Metrics: Parameter Specialization Score accuracy (MCQ) accuracy (open-ended generation) semantic entropy local intrinsic dimension Pearson correlation Spearman correlation
Baselines: FT-FV FT-CV FT-RV
Builds on: key-value memories in transformer MLPs knowledge superposition semantic entropy local intrinsic dimension
Closest papers by meaning
- Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws (ICLR 2025 · similarity 0.89)
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML 2024 · similarity 0.88)
- Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective (ICLR 2024 · similarity 0.88)
- Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models (ICLR 2024 · similarity 0.88)
- From Static to Dynamic: Knowledge Metabolism for Large Language Models (AAAI 2024 · similarity 0.88)
- Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.