Research map / Vision-language and multimodal / Multimodal LLMs
Mitigating Hallucination in Vision-Language Model with Depth and Spatial-aware Key-Value Refinement
Gusang Lee, Soohyun Kim, Donghoon Kim, Kyuhong Shim, Byonghyo Shim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies the representational origins of visual hallucination in vision-language models and proposes a depth- and spatial-aware key-value refinement to mitigate it.
Problem
Large vision-language models produce content not grounded in the input image (visual hallucination), and the representational origins of these hallucinations remain unclear.
Approach
Analyzes internal representations to show that successful grounding depends on depth and spatial information, and refines key-value representations using depth- and spatial-awareness to reduce hallucination. Full method details are unavailable beyond the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply DSCR depth and spatial KV cache refinement | LLaVA-1.5 · MME hallucination subset | total score: 892.68 -> 925.96 | Improved |
| apply DSCR | Qwen-VL · MME hallucination subset | total score: 825.01 -> 872.61 | Improved |
| apply DSCR | Qwen2.5-VL · MME hallucination subset | total score: 1042.91 -> 1039.97 | No clear effect |
| apply DSCR | LLaVA-1.5 · CHAIR | CHAIR_S: 48.2 -> 39.2 | Improved |
| apply DSCR | LLaVA-1.5 · CHAIR | Recall: 78.2 -> 73.4 | Hurt |
| apply DSCR | LLaVA-1.5 · RePOPE random | F1: 0.72 -> 0.80 | Improved |
| apply DSCR | mPLUG-Owl2 · POPE (GQA) random | accuracy: 0.85 -> 0.84 | Hurt |
| apply DSCR | LLaVA-1.5 · Depth Hallucination Mini-Benchmark | average score: 55.5 -> 78.5 | Improved |
| add DSCR on top of OPERA | OPERA · Depth Hallucination Mini-Benchmark | average score: 75.5 -> 76.0 | No clear effect |
| add DSCR on top of VCD | VCD · Depth Hallucination Mini-Benchmark | average score: 70.5 -> 72.0 | Improved |
| combine depth and spatial cues rather than one alone (ablation) | DSCR · MME hallucination subset | total score: 645.00 -> 650.00 | Improved |
| refine key cache in addition to value cache (ablation) | DSCR · MME hallucination subset | total score: 640.00 -> 650.00 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- HALC — HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding (ICML 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.94)
- Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention (CVPR 2025 · similarity 0.93)
- Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations (ICLR 2025 · similarity 0.93)
- Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.93)
- One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination (CVPR 2026 · similarity 0.93)
- Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models (CVPR 2026 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
