Research map / Large language models / Retrieval and knowledge
CollabEdit: Towards Non-destructive Collaborative Knowledge Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces collaborative knowledge editing (KE) across multiple privacy-separated clients and proposes CollabEdit, a non-destructive model-merging framework that approximates global MEMIT editing by sharing K K^T statistics.
Problem
Knowledge editing of LLMs with edit requests from multiple private parties aggregated via model merging is destructive: merged models lose much of the edit performance, and knowledge overlap, conflict, and forgetting arise across clients and rounds.
Approach
Derive a closed-form relation between the global MEMIT update and per-client updates, so that clients share only K_i K_i^T (privacy-preserving) and the server merges weighted local deltas with a coefficient matrix that mimics global editing; also proposes residual-based overlap detection, augmentation-based conflict resolution, and a dynamic covariance matrix against forgetting.
Key results
- On GPT-J with MEMIT over MCF, CollabEdit achieves Score 78.91 vs Global-Edit 79.03, while Task-Arithmetic reaches 63.44 and Simple-Average 54.22.
- On GPT2-XL with MEMIT over MCF, CollabEdit Score 77.18 vs Global-Edit 77.08, versus Ties-Merging 34.27.
- With dynamic covariance matrix after 1000 rounds of new edits on GPT-J MCF, Score drops only from 79.03 to 78.15 versus 74.68 with immutable C.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CollabEdit instead of Task-Arithmetic | MEMIT collaborative editing · MCF, GPT2-XL | Score: 60.85 -> 77.18 | Helped |
| Use CollabEdit instead of Task-Arithmetic | MEMIT collaborative editing · zsRE, GPT2-XL | Score: 30.45 -> 42.95 | Helped |
| Use CollabEdit instead of Task-Arithmetic | MEMIT collaborative editing · MCF, GPT-J (6B) | Score: 63.44 -> 78.91 | Helped |
| Use CollabEdit instead of Simple-Average | MEMIT collaborative editing · zsRE, GPT-J (6B) | Score: 40.22 -> 52.17 | Helped |
| Use CollabEdit rather than direct global editing | MEMIT editing · MCF, GPT2-XL | Score: 77.08 -> 77.18 | No clear effect |
| Use CollabEdit rather than direct global editing | MEMIT editing · zsRE, GPT-J (6B) | Score: 51.92 -> 52.17 | No clear effect |
| Use dynamic covariance matrix C instead of immutable C (ablation) | Continual editing with MEMIT · MCF, GPT-J (6B), after m rounds | Score: 74.68 -> 78.15 | Helped |
| Use dynamic covariance matrix C instead of immutable C (ablation) | Continual editing with MEMIT · zsRE, GPT-J (6B), after m rounds | Score: 41.21 -> 48.28 | Helped |
| Use dynamic covariance matrix C instead of immutable C (ablation) | Continual editing with MEMIT · MCF, GPT2-XL, after m rounds | Score: 64.80 -> 71.72 | Helped |
| Use dynamic covariance matrix C instead of immutable C (ablation) | Continual editing with MEMIT · zsRE, GPT2-XL, after m rounds | Score: 38.47 -> 39.80 | Helped |
| Add conflicting edit requests to the edited set | Collaborative editing with MEMIT · MCF, GPT-J | PA: 69.24 -> 27.83 | Hurt |
| Add conflicting edit requests to the edited set | Collaborative editing with MEMIT · MCF, GPT2-XL | EA: 58.28 -> 24.28 | Hurt |
Surprising findings
- Baselines achieve higher neighborhood scores (NS) than Global-Edit, which the authors attribute to under-fitting rather than better editing.
- Under MALMEN, CollabEdit's GPT2-XL zsRE score (35.11) is below Global-Edit (36.84).
Limitations
- Theoretical analysis is simplified to single-round editing; multi-round claim is argued by remark rather than proven.
- Privacy argument relies on empirical embedding inversion attack (GEIA) and the assumption of at least 2 edits per round.
- Conflict resolution and two-stage mechanism are only sketched and evaluated on one modified dataset without an automated conflict-detection stage.
Experimental setup
Datasets: Multi-CounterFact zsRE Easy dataset (Wikipedia-derived edits)
Benchmarks: Multi-CounterFact zsRE
Metrics: Efficacy Score Paraphrase Score Neighborhood Score Score (harmonic mean) Efficacy Accuracy Paraphrase Accuracy Neighborhood Accuracy Average Probability Difference Target Success Rate L2 norm of residual
Baselines: Simple-Average Task-Arithmetic TIES-Merging Global-Edit
Builds on: MEMIT MALMEN Task Arithmetic TIES-merging
Closest papers by meaning
- UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.87)
- Position: Editing Large Language Models Poses Serious Safety Risks (ICML 2025 · similarity 0.87)
- Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR 2024 · similarity 0.86)
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.86)
- AnyEdit: Edit Any Knowledge Encoded in Language Models (ICML 2025 · similarity 0.85)
- Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.