Research map / Vision-language and multimodal / Multimodal LLMs

Mitigating Manifold Departure: Uncertainty-Aware Subspace Rectification for Trustworthy MLLM Decoding

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MGAP, a training-free MLLM decoding method that projects final-layer hidden states onto an SVD-derived language-prior subspace from blind inputs and attenuates that component with an entropy- and mismatch-based gate to reduce hallucination while preserving orthogonal semantics.

Problem

MLLMs hallucinate objects inconsistent with the image due to over-reliance on language priors; existing training-free contrastive decoding suppresses priors indiscriminately, which can disrupt the hidden-state manifold and degrade performance.

Approach

Model language priors as a low-rank subspace from blind hidden states (SVD) and apply a selective, uncertainty-gated, consistency-weighted attenuation only of the projected prior component, leaving orthogonal semantic components unchanged, instead of a global linear contrastive shift.

Key results

• On POPE Adversarial with LLaVA v1.5 7B, MGAP reaches 84.59 accuracy and 84.46 F1 versus 80.16 / 82.05 for vanilla decoding.
• On CHAIR with LLaVA v1.5 7B, MGAP lowers CHAIRs from 47.4 (vanilla) to 26.2 and CHAIRi from 23.5 to 7.6.
• Inference time is 1.05 s/sample versus 2.14 s/sample for VCD on 3000 samples, roughly halving latency relative to contrastive baselines.

Surprising findings

• VCD underperforms vanilla decoding on POPE even on non-adversarial splits with LLaVA v1.5 7B.
• Ablated variants without protection or gating reach very high precision on LLaVA but with much lower F1 and accuracy, indicating over-conservative answering.

Limitations

• Relies on hidden-state subspace estimated from only 50 blind prompts with a fixed K=5, and sensitivity is only tested on LLaVA v1.5 7B.
• Theoretical guarantees rely on assumptions (prior-aligned error, access to h_gt) that are not verified at inference time.
• Manifold departure is defined via kNN distance on a reference bank, a proxy rather than a directly computed manifold.

Experimental setup

Datasets: COCO POPE CHAIR AMBER

Benchmarks: POPE CHAIR AMBER

Metrics: Accuracy Precision F1 CHAIRs CHAIRi Cover Hal Cog off-manifold sample count latency

Baselines: Vanilla VCD ICD HalTrapper DeCo MoD CODE

Builds on: Visual Contrastive Decoding contrastive decoding

Closest papers by meaning

• MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation (ICLR 2025 · similarity 0.89)
• HulluEdit: Single-Pass Evidence-Consistent Subspace Editing for Mitigating Hallucinations in Large Vision-Language Models (CVPR 2026 · similarity 0.89)
• Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding (ICML 2026 · similarity 0.88)
• Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding (NeurIPS 2025 · similarity 0.88)
• Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation (ICLR 2026 · similarity 0.88)
• Adaptive Residual-Update Steering for Low-Overhead Hallucination Mitigation in Large Vision-Language Models (ICML 2026 · similarity 0.88)