Research map / Vision-language and multimodal / Multimodal LLMs
BalancEdit: Dynamically Balancing the Generality-Locality Trade-off in Multi-modal Model Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BalancEdit, a training-free-of-backbone multi-modal model editing method that stores edits in a discrete codebook with dynamically sized influence radii estimated from rephrased-text positives and black-image negatives, and introduces the OKEDIT benchmark to measure the generality-locality trade-off.
Problem
Existing multi-modal model editing methods ignore that each fact has its own influence scope, causing poor balance between generality (correct answers on related inputs) and locality (unchanged answers on unrelated inputs).
Approach
Wrap one layer of a VLM with a codebook of keys, edited transformations and per-key radii; the radius is interpolated between the distance to a rephrased-text positive sample and to a black-image negative sample, so each edit gets its own influence scope without modifying backbone weights.
Key results
- On OKEDIT with BLIP-2 OPT, BalancEdit reaches HM 71.85 vs 51.70 for MEND and 45.69 for GRACE, with 100% edit accuracy.
- On MMEDIT with MiniGPT-4, BalancEdit achieves HM 88.08 vs 85.43 for MEND.
- Sequential editing over 50 edits on OKEDIT with MiniGPT-4 keeps edit accuracy at 100% and HM 65.95, still above baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BalancEdit instead of MEND | MiniGPT-4 editing · OKEDIT | HM: 61.07 -> 71.58 | Helped |
| Use BalancEdit instead of MEND | BLIP-2 OPT editing · OKEDIT | HM: 51.70 -> 71.85 | Helped |
| Use BalancEdit instead of MEND | MiniGPT-4 editing · MMEDIT | HM: 85.43 -> 88.08 | Helped |
| Use BalancEdit instead of MEND | BLIP-2 OPT editing · MMEDIT | HM: 85.29 -> 89.14 | Helped |
| Apply 50 edits sequentially rather than non-sequentially | BalancEdit with MiniGPT-4 · OKVQA, 50 sequential edits | HM: 71.07 -> 65.95 | Hurt |
| Replace black-image negative anchor with random negative samples (ablation) | BalancEdit with BLIP-2 OPT · OKEDIT | HM: 71.85 -> 65.61 | Hurt |
| Replace black-image negative anchor with random negative samples (ablation) | BalancEdit with MiniGPT-4 · OKEDIT | HM: 71.58 -> 64.08 | Hurt |
| Replace Euclidean distance with cosine distance (ablation) | BalancEdit · MMEDIT | HM: 88.08 -> 90.01 | Helped |
| Replace Euclidean distance with cosine distance (ablation) | BalancEdit · OKEDIT | HM: 71.58 -> 65.95 | Hurt |
| Replace black negative anchor with white negative anchor (ablation) | BalancEdit · OKEDIT subset | HM: 72.76 -> 73.23 | Helped |
| Edit layer 30 instead of layer 31 (ablation) | BalancEdit with MiniGPT-4 · OKEDIT | HM: 71.58 -> 73.75 | Helped |
| Edit layer 30 instead of layer 31 (ablation) | BalancEdit with BLIP-2 OPT · OKEDIT | HM: 71.85 -> 66.70 | Hurt |
Surprising findings
- Finetuning and IKE achieve near-perfect generality but locality collapses to under 20% on OKEDIT, showing that high edit success hides over-generalization.
- Using a white image instead of a black image as negative anchor yields similar HM (73.23 vs 72.76).
Limitations
- Similarity search across layers slows inference.
- Multi-hop editing is hard for key-based lookup since the influence scope is not fine-grained.
- Image generality remains weaker than locality on OKEDIT (I-Gen 65-76 vs Loc 53-61 trade-off), and alpha must be tuned on a held-out set per model.
Experimental setup
Datasets: OKVQA VQA-v2 MMEDIT
Benchmarks: OKEDIT MMEDIT
Metrics: Edit accuracy T-Gen I-Gen Locality harmonic mean
Baselines: Finetuning (FT) IKE MEND GRACE
Builds on: GRACE MEND IKE MMEDIT
Closest papers by meaning
- Modality-Decoupled Online Recursive Editing (ICML 2026 · similarity 0.87)
- UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.87)
- Rethinking Residual Distribution in Locate-then-Edit Model Editing (NeurIPS 2025 · similarity 0.87)
- Spectral Editing of Activations for Large Language Model Alignment (NeurIPS 2024 · similarity 0.86)
- Should We Really Edit Language Models? On the Evaluation of Edited Language Models (NeurIPS 2024 · similarity 0.86)
- Towards Lifelong Model Editing via Simulating Ideal Editor (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.