Research map / Vision-language and multimodal / Multimodal LLMs
Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs
Chongjun Tu, P.D. Ye, Dongzhan Zhou, Lei Bai, Gang Yu, Tao Chen, Wanli Ouyang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply attention reallocation (CHAIR-aligned) to greedy decoding | LLaVA-1.5-7B · CHAIR MSCOCO | F1: 77.1 -> 78.3 | Improved |
| apply attention reallocation (F1-aligned) to greedy decoding | LLaVA-1.5-7B · CHAIR MSCOCO | CHAIR_S: 47.0 -> 23.0 | Improved |
| apply attention reallocation to beam search | LLaVA-1.5-7B · CHAIR MSCOCO | CHAIR_S: 51.8 -> 21.6 | Improved |
| apply attention reallocation to nucleus sampling | LLaVA-1.5-7B · CHAIR MSCOCO | F1: 72.7 -> 75.0 | Improved |
| apply attention reallocation to mPLUG-Owl2 greedy decoding | mPLUG-Owl2 · CHAIR MSCOCO | CHAIR_S: 59.0 -> 19.8 | Improved |
| apply attention reallocation to Qwen2-VL | Qwen2-VL · CHAIR MSCOCO | CHAIR_S / CHAIR_I: 30.0 / 8.9 -> 21.0 / 5.9 | Improved |
| apply attention reallocation to MiniGPT-4 greedy (F1-aligned) | MiniGPT-4 · CHAIR MSCOCO | CHAIR_S: 24.4 -> 27.8 | Hurt |
| apply attention reallocation to nucleus sampling at aligned F1 | LLaVA-1.5-7B · CHAIR MSCOCO | CHAIR_S: 42.0 -> 42.2 | No clear effect |
| apply attention reallocation on AMBER (greedy) | LLaVA-1.5-7B · AMBER generative | CHAIR: 6.1 -> 3.2 | Improved |
| apply attention reallocation on AMBER (greedy) | LLaVA-1.5-7B · AMBER generative | Hal: 28.9 -> 19.2 | Improved |
| apply attention reallocation on AMBER (greedy) | MiniGPT-4 · AMBER generative | Cover: 62.6 -> 56.7 | Hurt |
| apply attention reallocation on GPT-4o evaluation | LLaVA-1.5-7B · GPT-4o assisted evaluation | correctness: 3.22 -> 3.40 | Improved |
Closest papers by meaning
- MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation (ICLR 2025 · similarity 0.93)
- Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation (AAAI 2026 · similarity 0.93)
- Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding (CVPR 2025 · similarity 0.93)
- Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs (NeurIPS 2025 · similarity 0.93)
- The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs? (NeurIPS 2025 · similarity 0.93)
- Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation (ICLR 2026 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
