Research map / Vision-language and multimodal / Multimodal LLMs

SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Preference tuning against hallucination · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a stylized-image hallucination benchmark (COCO photos with InstantStyle-generated Cartoon/Game/Graffiti/Painting/Sketch versions) and proposes SAVER, a training-free decoding method that revises final-layer logits with a selected early layer's visual-grounding logits.

Problem

LVLM object hallucination is substantially worse on stylized images (cartoon, game, sketch, etc.) than on photographs, and existing decoding-based mitigation methods degrade on such images.

Approach

Style-Aware Score measures how much early-layer logits over visual token positions support candidate tokens; SAVER picks the layer with maximal score per decoding step and adds its masked logits to the final-layer logits, scaled by that confidence, to correct hallucinated tokens without training.

Key results

• On Style-CHAIR with LLaVA-1.5-style setup, SAVER reaches average CHAIRi/CHAIRs 8.1/27.3 on InstructBLIP versus 8.9/28.1 for Deco (best baseline), and 8.3/27.3 on MiniGPT-4.
• On Style-POPE (MiniGPT-4, InstructBLIP, LLaVA-1.5), SAVER reports the highest average F1 among compared methods in the MiniGPT-4 and InstructBLIP tables (e.g., 67.58 vs 67.05 for Deco on MiniGPT-4 adversarial F1).
• On AMBER with LLaVA-1.5-7b, SAVER obtains existence F1 80.7 and attribute Acc 68.3 versus Deco's 62.5 and 67.2.

Surprising findings

• Converting stylized images back to photographic style with a style-transfer model still yields high hallucination (Ci/Cs 14.9/46.0 stylized vs 14.3/34.0 photographic), suggesting the style itself, not only transfer noise, drives hallucination.
• DoLa performs very poorly on some Style-POPE settings (e.g., MiniGPT-4 adversarial accuracy around 14% for sketch) and shows high CHAIR scores, unlike other decoding methods.

Limitations

• Only four open LVLMs (LLaVA-1.5, LLaVA-1.6, MiniGPT-4, InstructBLIP) are used for the mitigation experiments; closed models are only benchmarked.
• Style dataset is limited to 1,800 images across five styles generated by a single style-transfer model.
• Gains on LLaVA-1.5 CHAIR are only competitive with Deco rather than clearly best.

Experimental setup

Datasets: COCO Style-CHAIR Style-POPE MME AMBER

Benchmarks: Style-CHAIR Style-POPE MME AMBER Real-World (depth, thermal, medical)

Metrics: CHAIRi CHAIRs F1 ACC Precision Recall BLEU-1/2/3/4 METEOR ROUGE-L MME total score

Baselines: Greedy decoding Beam search DoLa OPERA Deco AGLA

Builds on: DoLa OPERA Deco AGLA POPE CHAIR

Closest papers by meaning

• Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.93)
• Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.93)
• Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling (NeurIPS 2025 · similarity 0.92)
• Image Token Matters: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing (NeurIPS 2025 · similarity 0.92)
• Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy (ICML 2026 · similarity 0.92)
• Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations (ICLR 2025 · similarity 0.92)