AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Anchor-Final Self-Supervision Drives Hallucination-Aware Optimization in Large Vision-Language Models

Jiaxi Liu, Yifeng Yang, Xinbing Wang, Qinying Gu, Nanyang Ye

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AFS (Anchor-Final Self-Supervision), a framework that uses discrepancies between intermediate-layer and final-layer predictions of an LVLM to selectively apply self-supervision for hallucination-aware optimization.

Problem

Large vision-language models often generate tokens that are not aligned with visual evidence (hallucinations).

Approach

Uses the disagreement between intermediate ("anchor") and final layer predictions as a self-supervision signal, applied selectively to visual tokens, to optimize the model against hallucinations without external annotations.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add visual-selective gating (ablation)AFS reward · CHAIRCHAIR_S: 26.2 -> 24.6Improved
add hallucination-aware token classification (ablation)AFS reward · CHAIRCHAIR_S: 27.2 -> 24.6Improved
add hallucination-aware token classification (ablation)AFS reward · CHAIRCHAIR_I: 8.8 -> 7.7Improved
add anchor-final distributional consistency (ablation)AFS reward · CHAIRCHAIR_S: 25.6 -> 24.6Improved
add anchor-final distributional consistency (ablation)AFS reward · CHAIRRecall: 63.9 -> 64.6Improved
add token-specific weight (ablation)AFS reward · CHAIRCHAIR_S: 25.8 -> 24.6Improved
add hallucination-aware token classification (ablation)AFS reward · CHAIRRecall: 65.7 -> 64.6Hurt
use anchor layers 20-27 instead of 14-21 (ablation)AFS · CHAIRCHAIR_S: 25.8 -> 24.6Improved
use anchor layers 20-27 instead of 17-24 (ablation)AFS · CHAIRCHAIR_S: 25.6 -> 24.6Improved
set reward sharpness alpha 1.0 instead of 0.5 (ablation)AFS · CHAIRCHAIR_S: 25.0 -> 24.6Improved
set reward sharpness alpha 1.0 instead of 2.0 (ablation)AFS · CHAIRCHAIR_S: 25.4 -> 24.6Improved
train with AFS self-supervised reward instead of standard GRPO with binary GQA rewardQwen2.5-VL-7B · CHAIRCHAIR_S: 29.4 -> 24.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy