Research map / Large language models / Retrieval and knowledge
Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UnKE, a knowledge editing method for unstructured long-form knowledge that uses non-local block key-value storage and cause-driven (last-token) optimization, evaluated on a new UnKEBench benchmark against locate-then-edit and fine-tuning baselines.
Problem
Existing knowledge editing methods target structured triples and rely on local MLP key-value storage and subject-term localization, which fail for long-form, noisy, unstructured knowledge.
Approach
Extends key-value storage from single MLP layers to transformer blocks (shallow layers as key generator, deep layers as value generator) and edits only the last token of the input with preservation and causal losses, removing the need to locate subject terms.
Key results
- On UnKEBench with LLaMA2-7B-Chat, UnKE reaches 99.61 Bert-Score (original questions) versus 75.90 for MEMIT and 76.52 for ROME.
- On KEBench structured editing, UnKE improves Ori-Acc and Para-Acc by 16.4 and 18 points over strong baselines (reported in the text).
- Human evaluation gives UnKE 4.78 correctness versus 3.39 for ROME and 3.25 for MEMIT on a 1-5 scale.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UnKE instead of LoRA | LLaMA2-7B-Chat · UnKEBench | Bert-Score: 99.61 / 93.09 vs 88.05 / 84.62 | Helped |
| use UnKE instead of MEMIT | Qwen1.5-7B-Chat · UnKEBench | FactScore: 17.81 vs 40.08 | Helped |
| use UnKE instead of ROME | knowledge editing · KEBench | Ori-Acc: 77.90 -> 94.30 | Helped |
| use UnKE instead of ROME | knowledge editing · KEBench | Para-Acc: 68.40 -> 86.40 | Helped |
| optimize only MLP parameters instead of both MLP and Attention (ablation) | UnKE · UnKEBench | Bert-Score: 99.61 / 93.09 -> 95.43 / 87.87 | Hurt |
| optimize only Attention parameters instead of both MLP and Attention (ablation) | UnKE · UnKEBench | Bert-Score: 99.61 / 93.09 -> 92.66 / 81.62 | Hurt |
| add Key Preservation Loss (ablation) | UnKE · UnKEBench | MMLU: 29.44$_{\mathrm{0.34}\downarrow}$ -> 29.68$_{\mathrm{0.10}\downarrow}$ | Helped |
| add Key Causal Loss (ablation) | UnKE · UnKEBench | Fact-Score: 16.77 -> 42.49 | Helped |
| add both Key Preservation Loss and Key Causal Loss (ablation) | UnKE · UnKEBench | Bert-Score: 9.32 / 9.79 -> 99.61 / 93.09 | Helped |
| set optimization layer L to 11 rather than L=7 (ablation) | UnKE · UnKEBench | Bert-Score: 99.61 / 93.09 -> 90.04 / 86.73 | Hurt |
| increase batch size (ablation) | UnKE · UnKEBench paraphrased questions | Bert-Score: 93.09 -> 91.55 -> 89.98 -> 88.33 -> 85.98 -> 84.07 -> 85.21 | Hurt |
Surprising findings
- Editing only layer 7 versus editing 5 layers (MEMIT) shows little effect on structured success rate or unstructured Bert-Score, suggesting knowledge is distributed non-locally rather than in specific layers.
- IKE achieves the highest FactScore (94.60) but the authors argue this reflects reading comprehension rather than true parameter editing.
Limitations
- Human evaluation covers only 36 samples per method with moderate inter-annotator agreement (Fleiss' kappa 0.57).
- UnKE modifies more parameters than ROME/MEMIT, which may slightly increase impact on unrelated knowledge (Loc-FactScore on LLaMA2 is similar to baselines, but Qwen results are mixed).
- Structured-editing results on KEBench and KnowEdit are mixed; UnKE is not uniformly best on KnowEdit (e.g., ZsRE and WikiData_counterfact slightly lower than ROME/MEMIT).
Experimental setup
Datasets: UnKEBench ConflictQA Counterfact KEBench KnowEdit MMLU Wikipedia
Benchmarks: UnKEBench KEBench KnowEdit MMLU
Metrics: BERTScore BLEU ROUGE-1 ROUGE-2 ROUGE-L FactScore MMLU Loc-FactScore human correctness/similarity/coherence
Baselines: FT-L FT-A ROME MEMIT MEND LoRA AdaLoRA RECT IKE
Builds on: ROME MEMIT EasyEdit IKE
Closest papers by meaning
- UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.89)
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.87)
- Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration (NeurIPS 2024 · similarity 0.86)
- Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.86)
- Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis (NeurIPS 2026 · similarity 0.86)
- History Matters: Temporal Knowledge Editing in Large Language Model (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.