AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Enhancing Retrieval-Augmented Large Vision Language Models via Knowledge Conflict Mitigation

Wenbin An, Jiahao Nie, Feng Tian, Mingxiang Cai, Yaqiang Wu, Xiaoqin Zhang, Shijian Lu

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Addresses knowledge conflicts between retrieved multimodal context and LVLMs' parametric knowledge in multimodal RAG for knowledge-intensive tasks.

Problem

Retrieved contextual knowledge in multimodal retrieval-augmented generation is often misaligned with LVLMs' internal parametric knowledge, causing knowledge conflicts that degrade answers in knowledge-intensive tasks.

Approach

Mitigates knowledge conflicts between retrieved multimodal context and an LVLM's parametric knowledge to make retrieval-augmented LVLMs more reliable; the specific mechanism is not described in the available abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add KCM to retrieval-augmented decodingLLaVA-1.5 · InfoSeekaccuracy: 51.97 -> 58.67Improved
add KCMLLaVA-1.5 · OK-VQAaccuracy: 46.17 -> 60.90Improved
add KCMLLaVA-1.5 · E-VQAaccuracy: 19.14 -> 29.95Improved
add KCMLLaVA-1.5 · Human free-formaccuracy: 7.74 -> 12.42Improved
add KCMInstructBLIP · Human free-formaccuracy: 4.02 -> 6.77Improved
add KCMShikra · InfoSeekaccuracy: 19.41 -> 27.90Improved
add KCMMiniGPT-4 · InfoSeekaccuracy: 25.83 -> 31.29Improved
add KCM with ground-truth contextLLaVA-1.5 · InfoSeekaccuracy: 65.5 -> 71.2Improved
add KCM with ground-truth contextLLaVA-1.5 · InfoSeekmemorization ratio: 46.9 -> 39.7Improved
add KCM with noisier 2nd retrieved knowledgeLLaVA-1.5 · InfoSeekaccuracy: 38.8 -> 44.9Improved
compare KCM against best prior decoding method VCDKCM · OK-VQAaccuracy: 57.80 -> 60.90Improved
add Attention Adjustment, Neuron Pruning, Context logits and Entropy weighting incrementally (ablation)KCM · InfoSeek and AOK-VQAVQA accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy