Research map / Large language models / Instruction and fine-tuning
Editing Memories Through Few Targeted Neurons
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TNF-DA, which fine-tunes only the ~1% "targeted neurons" in FFNs tied to an edited fact, with relation-based data augmentation to preserve locality in model editing.
Problem
Fine-tuning-based model editing in LLMs struggles to preserve irrelevant knowledge (locality) and tends to overfit or lose generalization/specificity, since irrelevant knowledge is hard to measure and constraining weight changes is not explicit enough.
Approach
Causal mediation analysis on GPT2 finds about 1% of FFN neurons contribute strongly to a fact, and facts sharing a relation share these neurons; the method edits only these targeted neurons and trains on a relation-based augmented dataset with a KL locality loss.
Key results
- TNF-DA reaches an editing score of 87.65 on GPT2-XL, versus 52.47 for FT, 38.70 for FT+W and 57.92 for MEND, with specificity 79.42.
- TNF-DA is comparable to locate-and-optimize methods ROME (90.99) and PMET (87.15) on the editing score, while ROME and PMET remain slightly higher on score.
- Ablation: editing random neurons instead of targeted neurons drops the editing score from 87.65 to 52.73.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TNF-DA instead of FT | GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Score: 52.47 -> 87.65 | Helped |
| use TNF-DA instead of FT | GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Specificity: 30.72 -> 79.42 | Helped |
| use TNF-DA instead of ROME | GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Score: 90.99 -> 87.65 | Hurt |
| use TNF-DA instead of ROME | GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Generalization: 95.10 -> 87.21 | Hurt |
| replace targeted-neuron parameters with randomly chosen parameters (ablation) | TNF-DA on GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Score: 87.65 -> 52.73 | Hurt |
| replace targeted-neuron parameters with randomly chosen parameters (ablation) | TNF-DA on GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Specificity: 79.42 -> 32.14 | Hurt |
| replace relation-based data augmentation with random data augmentation (ablation) | TNF-DA on GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Score: 87.65 -> 77.39 | Hurt |
| replace relation-based data augmentation with random data augmentation (ablation) | TNF-DA on GPT2-XL (1.5B) · COUNTERFACT (2K edits) | Specificity: 79.42 -> 56.89 | Hurt |
| add random prefixes during edit training (ablation) | TNF-DA without data augmentation · COUNTERFACT | Specificity: 26.98 -> 42.06 | Helped |
Surprising findings
- Editing a random 1% of neurons still gives high efficacy, but specificity drops sharply, showing that targeted neurons matter mainly for locality.
- The number of epochs needed to edit a fact did not show clear regularity by relation, undermining fixed-epoch training.
Limitations
- Does not handle reasoning-related knowledge, since it lacks knowledge-graph support and works at the semantic level.
- Being fine-tuning based, editing efficiency is hard to improve through closed-form optimization like ROME/MEMIT because targeted neurons are spread across layers.
- Ignores negative-contribution neurons and neuron interactions.
Experimental setup
Datasets: COUNTERFACT
Benchmarks: COUNTERFACT
Metrics: Efficacy Generalization (paraphrase) Specificity (neighborhood) Editing score (harmonic mean) Fluency Consistency
Baselines: FT FT+W MEND ROME MEMIT PMET
Builds on: ROME knowledge neurons constrained fine-tuning causal mediation analysis
Closest papers by meaning
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs (ICLR 2025 · similarity 0.93)
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (NeurIPS 2024 · similarity 0.91)
- Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs (NeurIPS 2025 · similarity 0.90)
- Perturbation-Restrained Sequential Model Editing (ICLR 2025 · similarity 0.89)
- Should We Really Edit Language Models? On the Evaluation of Edited Language Models (NeurIPS 2024 · similarity 0.88)
- Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.