Research map / Vision-language and multimodal / Multimodal LLMs
VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds VLKEB, a multimodal knowledge-editing benchmark for LVLMs using MMKG-linked real images, adds a multi-hop Portability metric, and benchmarks FT, KE, IKE, SERAC and MEND on five LVLMs in single and sequential editing.
Problem
Knowledge editing for large vision-language models lacks high-quality benchmarks: the existing MMEdit uses synthesized images for generality and does not evaluate whether edited knowledge is applied in related content (Portability).
Approach
Construct image editing data from a multimodal knowledge graph (MMKG) with real, manually checked image pairs, filter similar-entity images for harder image locality, and extend Portability to one-hop and multi-hop reasoning chains over KG triples.
Key results
- IKE achieves the highest Portability on most models, e.g. 63.33 on LLaVA-1.5 and 57.99 on Qwen-VL, with near-100% reliability and generality.
- MEND gives the best image locality among methods (e.g. 90.14 on LLaVA-1.5), while SERAC reaches only about 1-5% image locality.
- Editing a one-hop triple raises Portability for all methods, e.g. Qwen-VL FT (LLM) rises from 16.15 to 88.88.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IKE instead of parameter-update methods for single editing | LVKEB single editing across five LVLMs · VLKEB single editing | Portability: 44.22; 54.30; 63.33; 57.99 | Helped |
| Use FT (Vis) instead of other methods | mPLUG-Owl2 single editing · VLKEB single editing, mPLUG-Owl2 | Portability: 74.09 | Helped |
| Use MEND instead of other editing methods | single editing across five LVLMs · VLKEB single editing | Image Locality: 89.05; 92.67; 90.14; 78.65; 90.47 | Helped |
| Add the corresponding one-hop knowledge edit (ablation) | FT (LLM), BLIP2-OPT · VLKEB one-hop Portability, BLIP2-OPT | Portability: 17.13→46.71 (↑ 29.58) | Helped |
| Add the corresponding one-hop knowledge edit (ablation) | FT (LLM), MiniGPT-4 · VLKEB one-hop Portability, MiniGPT-4 | Portability: 27.13→44.65 (↑ 17.52) | Helped |
| Add the corresponding one-hop knowledge edit (ablation) | FT (LLM), LLaVA-1.5 · VLKEB one-hop Portability, LLaVA-1.5 | Portability: 30.23→74.79 (↑ 44.56) | Helped |
| Add the corresponding one-hop knowledge edit (ablation) | FT (LLM), Qwen-VL · VLKEB one-hop Portability, Qwen-VL | Portability: 16.15→88.88 (↑ 72.73) | Helped |
| Add the corresponding one-hop knowledge edit (ablation) | FT (Vis), mPLUG-Owl2 · VLKEB one-hop Portability, mPLUG-Owl2 | Portability: 74.09→93.80 (↑ 19.71) | Helped |
| Increase the number of reasoning hops | edited LVLMs across editing methods · VLKEB multi-hop Portability | Portability | Hurt |
| Fine-tune LLM layers rather than use the unedited base model | LVLMs in sequential editing · VLKEB sequential editing, larger test gaps | Reliability: 99.95; 57.52; 54.84; 51.25; 47.18 | Hurt |
| Use MEND for sequential editing | BLIP2-OPT, MiniGPT-4, and LLaVA-1.5 · VLKEB sequential editing | Reliability: 50 edits; around 14 edits; around 20 edits | Hurt |
| Fine-tune LLM layers rather than vision modules | LVLMs in sequential editing · VLKEB sequential editing | Locality | Hurt |
Surprising findings
- SERAC's Portability gains after editing one-hop knowledge reflect retrieval of the 1-hop QA rather than the first edit being applied, i.e. a 'cheat sheet' effect.
- MEND collapses to NaN logits after a small number of sequential edits (about 14 to 50 depending on model).
Limitations
- Evaluates only LLM-derived editing methods adapted to LVLMs, not methods designed for multimodal interaction.
- Portability questions are generated from KG triples and may not cover all related content.
- Experiments are limited to five LVLMs with 7-10B-scale language backbones.
Experimental setup
Datasets: MMKG FB15K DB15K NQ
Benchmarks: VLKEB
Metrics: Reliability Text Generality Image Generality Text Locality Image Locality Portability
Baselines: FT (LLM) FT (Vis) KE IKE SERAC MEND
Builds on: MMEdit Knowledge Editor MEND SERAC IKE
Closest papers by meaning
- Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit (AAAI 2025 · similarity 0.92)
- MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge (ICLR 2025 · similarity 0.92)
- Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.92)
- Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts (CVPR 2025 · similarity 0.91)
- Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.89)
- MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.