AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Self-Augmented Visual Contrastive Decoding

Eun Woo Im, Muhammad Kashif Ali, Vivek Gupta

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Self-Augmented Visual Contrastive Decoding, a method that mitigates hallucination in LVLMs by building contrastive visual inputs from the text query context rather than generic visual augmentations.

Problem

LVLMs inherit hallucination tendencies from their language models; existing visual contrastive decoding uses generic visual augmentations that ignore the text query's context, limiting effectiveness.

Approach

Replaces generic visual augmentations used in visual contrastive decoding with query-aware self-augmented visual inputs, so the contrastive negative reflects the specific context of the text query.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use query-adaptive augmentation (Self-Aug) instead of a single static augmentation (ablation)Self-Aug · MME-Perception (LLaVA-1.5-7B)score: 1351.76 -> 1431.30 (best static noise vs Self-Aug)Improved
replace APC with SAT thresholding (ablation)Self-Aug · MME-Perception (LLaVA-1.5-7B)score: 1345.46 -> 1431.30Improved
replace APC with SAT thresholding (ablation)VCD · MME-Perception (LLaVA-1.5-7B)score: 1323.67 -> 1395.17Improved
add operational knowledge (augmentation definitions) to the SAS prompt (ablation)SAS prompting · MME-Perception (LLaVA-1.5-7B)score: 1312.39 -> 1419.08Improved
add reasoning before the augmentation choice (ablation)SAS prompting · MME-Perception (LLaVA-1.5-7B)score: 1419.08 -> 1428.02No clear effect
add in-context examples to the SAS prompt (ablation)SAS prompting · MME-Perception (LLaVA-1.5-7B)score: 1419.08 -> 1428.63No clear effect
use inverse-entropy threshold (Hdecay) rather than direct-entropy or APC (ablation)SAT · MME-Perception (LLaVA-1.5-7B)scoreImproved
use Self-Aug versus multinomial sampling on Qwen3-VL-8BQwen3-VL-8B · MME-Perceptionscore: 1725.16 -> 1726.77No clear effect
apply Self-Aug versus multinomial sampling on InstructBLIPInstructBLIP-7B · MMVPaccuracy: 19.20 -> 16.13Hurt
use larger 13B model for augmentation selection (ablation)Self-Aug · Agreement with Oracleagreement: 64.15% -> 66.19%Improved
use full reasoning+ICL prompt versus lightweight prompt (ablation)Self-Aug · LLaVA-Bench (LLaVA-1.5-13B)score: 73.82 -> 76.24Improved
use Self-Aug instead of VACoDe on POPE-MSCOCO accuracyLLaVA-1.5-7B · POPE-MSCOCOaccuracy: 84.29 -> 82.93Hurt

Experimental setup

Baselines: visual contrastive decoding

Builds on: visual contrastive decoding

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy