Research map / Vision-language and multimodal / Multimodal LLMs

First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Preference tuning against hallucination · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes First Logit Boosting (FLB), a training-free decoding method that adds the first generated token's logit, with a growing weight, to later decoding steps to counter long-term visual decay and reduce object hallucination in LVLMs.

Problem

Object hallucination in large vision-language models worsens as generation proceeds (long-term decay of visual grounding), and existing contrastive decoding methods fail to address this while doubling inference cost.

Approach

Store the logit of the first generated token (conditioned on image and prompt) and add it to every subsequent step with a time-increasing weight w_t = gamma(1-exp(-lambda t)), constrained by an adaptive plausibility filter; this needs only one forward pass and yields a "The" effect that stabilizes noun prediction.

Key results

• On AMBER with LLaVA-1.5, FLB reduces CHAIR from 11.5 (baseline) to 6.1 and Hal from 48.9 to 31.6, with Cover roughly unchanged (50.4 vs 50.1).
• On CHAIR with LLaVA-1.5, FLB lowers CHAIR_s from 57.5 to 43.5 while keeping recall at 73.6 (baseline 73.3).
• FLB runs at near-baseline speed, whereas VCD, ICD, and M3ID run about twice as slow as baseline.

Surprising findings

• Much of the gain is attributed to the "The" effect: boosting only the first-token "The" logit gives nearly the same hallucination reduction as the full method (Hal 29.9 vs 30.7).
• On discriminative benchmarks (POPE, MME), FLB matches the beta-only ablation, indicating little effect on short outputs.

Limitations

• FLB does not eliminate the underlying long-term decay caused by RoPE; it only mitigates it via early-token reinforcement.
• Injects a fixed visual signal from the first token rather than adapting to evolving visual semantics.
• Under greedy decoding, Cover slightly decreases; increases the frequency of sentence-initial 'The'.

Experimental setup

Datasets: AMBER MSCOCO 2014 validation (CHAIR) POPE MME MMHalBench ConvBench

Benchmarks: AMBER CHAIR MMHalBench ConvBench POPE MME

Metrics: CHAIR Cover Hal Cog Recall accuracy F1 GPT-4V score win rate

Baselines: VCD ICD M3ID Baseline decoding

Builds on: Visual Contrastive Decoding Contrastive decoding Rotary Positional Embedding

Closest papers by meaning

• Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance (ICML 2025 · similarity 0.92)
• Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.92)
• The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models Via Visual Information Steering (ICML 2025 · similarity 0.92)
• Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation (AAAI 2026 · similarity 0.91)
• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · similarity 0.91)
• Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination (CVPR 2026 · similarity 0.91)