Research map / Vision-language and multimodal / Multimodal LLMs

Uncovering and Mitigating Transient Blindness in Multimodal Model Editing

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes De-VQA, a dynamic locality evaluation framework for multimodal model editing with seven data types across RI/NI/CI locality, reveals "transient blindness" (edits make models over-rely on text), and mitigates it with a KL-based cross-modal locality loss added to MEND.

Problem

Existing multimodal model editing locality metrics only test low-similarity or random inputs and check output preservation, missing overfitting to edit-similar text and reduced reliance on visual inputs after editing.

Approach

Build a Cartesian-product evaluation over retrieved similar text and image pairs to define random-image, no-image, and consistent-image locality metrics; attribute the failure to imbalanced text vs. image token contributions after editing; and add a multimodal KL locality loss (RI, CI, NI inputs) on top of MEND to rebalance cross-modal updates.

Key results

• On De-VQA, the proposed method reaches 0.7340 mean on MiniGPT4 VQA and 0.5897 on Blip2OPT VQA, versus 0.3768 for MEND on Blip2OPT VQA.
• The method reports roughly 0.7 on NI-Loc and CI-Loc and 0.6 on RI-Loc, with a claimed 17% average locality improvement.
• Qwen-VL ablation: RI-loc rises from 0.195 to 0.5046 and CI-Loc from 0.099 to 0.5151 with the added locality loss, at a cost of 3.96% in-scope accuracy.

Surprising findings

• Methods such as SERAC, LiveEdit, and LEMoE score very low on NI-Loc (<0.3), RI-Loc (<0.1), and CI-Loc (<0.2) despite near-perfect scores on traditional metrics.
• Editing only the visual module (FT-V) yields a perfect NI-Loc of 1 because text-only inputs are unaffected, showing that some locality scores are trivially satisfied.

Limitations

• Evaluation sample selection is tied to the IKE retriever and top-1 text similarity, so the benchmark depends on a specific retrieval procedure.
• Mitigation is built on MEND and only the locality loss is novel; results on T2I1 remain limited for fine-grained entity distinctions.
• Only one edit at a time is evaluated, with one sample per locality type.

Experimental setup

Datasets: VQA (EVQA) VLKEB

Benchmarks: VQA VLKEB De-VQA

Metrics: Reliability T-Gen I-Gen T-Loc I-Loc RI-Loc NI-Loc CI-Loc

Baselines: FT MEND SERAC TP LTE RECIPE LEMoE LiveEdit HICE

Builds on: MEND IKE EasyEdit ComprehendEdit

Closest papers by meaning

• MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge (ICLR 2025 · similarity 0.88)
• Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit (AAAI 2025 · similarity 0.88)
• ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing (AAAI 2025 · similarity 0.87)
• TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs (NeurIPS 2026 · similarity 0.87)
• Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · similarity 0.87)
• Modality-Decoupled Online Recursive Editing (ICML 2026 · similarity 0.86)