Research map / Large language models / Retrieval and knowledge
Knowledge Localization: Mission Not Accomplished? Enter Query Localization!
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Re-examines the knowledge localization assumption of the knowledge neuron thesis, finds widespread inconsistent knowledge, and proposes the Query Localization assumption plus a Consistency-Aware KN modification method.
Problem
The Knowledge Localization assumption underlying the Knowledge Neuron thesis may be too rigid for knowledge storage and ignores the attention module's role in knowledge expression in LLMs.
Approach
Shows that many facts (inconsistent knowledge) have inconsistent knowledge neurons across paraphrased queries, proposes Query Localization (query-KN mapping and dynamic KN selection via attention-based knowledge synapses), and uses a consistency-aware score to select neurons for model editing.
Key results
- For LLaMA3-8b, 77% of facts are classified as inconsistent knowledge under the static threshold, with a t-statistic of 123 (p < 1e-6) between consistent and inconsistent groups.
- Consistency-aware KN modification reaches Avg 0.51 on inconsistent knowledge for LLaMA3-8b under erasure, with Delta PPL 0.09, versus 0.42 and 1.05 for the baseline neuron sets.
- The paper reports about 8% and 9% improvements over two baselines in the erasure setting on LLaMA3-8b.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Consistency-Aware Score-based neuron selection instead of query-specific KNs (Ni) | Knowledge modification on K_I · ParaRel, Erasure, LLaMA3-8b | Avg: 0.43 -> 0.51 | Helped |
| Use Consistency-Aware Score-based neuron selection instead of the union of query KNs (Nu) | Knowledge modification on K_I · ParaRel, Erasure, LLaMA3-8b | Δ PPL: 1.05 -> 0.09 | Helped |
| Use QL-based Consistency-Aware KN modification instead of KL (Ni) | Sequential editing on LLaMA3-8b · 100 randomly sampled facts | Avg: 0.29±0.16 -> 0.42±0.18 | Helped |
| Use QL-based Consistency-Aware KN modification instead of KL (Ni) | Sequential editing on LLaMA3-8b · 100 randomly sampled facts | Δ PPL: 0.37±0.24 -> 0.29±0.18 | Helped |
Surprising findings
- Facts are often inconsistent: most facts in LLaMA3-8b fail the knowledge localization assumption, so consistent knowledge is the minority.
- Suppressing attention-based knowledge synapses sharply reduces knowledge neuron values, indicating attention selects KNs rather than storing knowledge.
Limitations
- Evaluation uses only a small subset (one datum per relation, 36 test points) due to compute cost, and only GPT-2 and LLaMA models were tested.
- Only the query-KN mapping aspect of QL is applied to editing; dynamic KN selection is not integrated into the method.
- Thresholds are set by hand (static 0.1) or via Otsu, and the consistency score relaxation is a heuristic.
Experimental setup
Datasets: ParaRel WikiText2
Benchmarks: ParaRel WikiText2
Metrics: Reliability Generalization Locality Avg Delta PPL Consistency Score Delta Prob t-statistic
Baselines: Integrated Gradients knowledge localization Sequential Integrated Gradients Architecture-adapted Multilingual Integrated Gradients KL-based neuron selection (N_i) union of KNs (N_u)
Builds on: Knowledge Neuron thesis integrated gradients knowledge neuron model editing
Closest papers by meaning
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs (ICLR 2025 · similarity 0.91)
- Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons (AAAI 2024 · similarity 0.91)
- Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts (AAAI 2025 · similarity 0.90)
- What does the Knowledge Neuron Thesis Have to do with Knowledge? (ICLR 2024 · similarity 0.89)
- Capability Localization: Capabilities Can be Localized rather than Individual Knowledge (ICLR 2025 · similarity 0.89)
- Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.