Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Blind Noising: Disentangled Visual Rectification for Hallucination Mitigation in MLLMs
Yujia Chen, Rui Sun, Bingzhou Wang, Huayu Mai, Wangkai Li, Zhaoyang Li, Aibing Li, Wenzhang SUN
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that noise-induced drift used by Visual Contrastive Decoding entangles structural degradation with hallucination-relevant signal, and proposes a disentangled visual rectification method to mitigate hallucinations in MLLMs.
Problem
Hallucinations in multimodal large language models; existing Visual Contrastive Decoding assumes the output shift from noise-perturbed images isolates the hallucination direction, which the paper argues is flawed for Language-Image Pretrained encoders.
Approach
Argues that noise-induced drift in LIP encoders is a coupled vector mixing structural degradation and hallucination signal, and disentangles these components to rectify visual representations rather than penalizing blind noise-based contrast.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DVR decoding | LLaVA-1.5-7B · CHAIR | CHAIRs: 54.2 -> 47.4 | Improved |
| add DVR decoding | LLaVA-1.5-7B · CHAIR | Recall: 72.5 -> 78.6 | Improved |
| use DVR instead of VCD | LLaVA-1.5-7B · POPE Adversarial | accuracy: 78.2 -> 80.2 | Improved |
| add VCD to Qwen3-VL-8B | Qwen3-VL-8B · CHAIR | CHAIRs: 21.0 -> 23.7 | Hurt |
| add DVR to Qwen3-VL-8B | Qwen3-VL-8B · CHAIR | CHAIRs: 21.0 -> 19.0 | Improved |
| add Stream I with DINOv3 auxiliary encoder (without Stream II) (ablation) | DVR · CHAIR | CHAIRs: 54.2 -> 49.2 | Improved |
| use DINOv3 instead of DINOv2 as auxiliary encoder (ablation) | DVR Stream I · CHAIR | CHAIRs: 50.4 -> 49.2 | Improved |
| use SigLIP (LIP paradigm) as auxiliary encoder instead of SSL (ablation) | DVR Stream I · CHAIR | CHAIRs: 54.2 -> 53.6 | No clear effect |
| add Stream II decoding contrast to Stream I (ablation) | DVR · CHAIR | CHAIRs: 49.2 -> 47.4 | Improved |
| add Stream I rectification to VCD-style contrast (ablation) | DVR · CHAIR | CHAIRs: 51.8 -> 47.4 | Improved |
| scale DVR from 7B to 13B (ablation) | DVR on LLaVA-1.5 · CHAIR | CHAIRs: 47.4 -> 45.4 | Improved |
| use DVR (7B) versus cross-scale VCD (13B-7B) (ablation) | LLaVA-1.5-7B · CHAIR | CHAIRs: 47.0 vs 47.4 | No clear effect |
Surprising findings
- Noise-induced drift in LIP encoders is a coupled vector entangling structural degradation with hallucination signal, so penalizing it as the hallucination direction is flawed.
Experimental setup
Baselines: Visual Contrastive Decoding
Builds on: Visual Contrastive Decoding
Closest papers by meaning
- ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models (AAAI 2025 · similarity 0.93)
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · similarity 0.92)
- Hallucination-aware Intermediate Representation Editing in Large Vision-Lanugage Models (ICLR 2026 · similarity 0.92)
- Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.92)
- Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.92)
- SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding (ICML 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
