Research map / Vision-language and multimodal / Multimodal LLMs
Taming the Phantom: Token-Asymmetric Filtering for Hallucination Mitigation in Large Vision-Language Models
Shuyi Ouyang, Hongyi Wang, Gongfan Fang, Xinyin Ma, Lanfen Lin, Xinchao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Token-level analysis of LVLM hallucination motivates a token-asymmetric filtering method to mitigate hallucinations (details unavailable beyond the abstract).
Problem
Hallucination in large vision-language models undermines reliability; prior work studied causes at the modality level, leaving token-level interaction patterns underexplored.
Approach
Conducts a token-level analysis of LVLM interactions beyond modality level and uses the identified asymmetry between tokens to filter them for hallucination mitigation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply TAF (phantom isolation + anchor emphasis) training-free | LLaVA-1.5-7B · POPE Adversarial | F1: 76.84 -> 86.21 | Improved |
| apply TAF vs best baseline AGLA | LLaVA-1.5-7B · CHAIR MSCOCO | CHAIRs: 47.6 -> 42.5 | Improved |
| apply TAF to mPLUG-Owl2 captioning | mPLUG-Owl2 · CHAIR MSCOCO | CHAIRs: 53.7 -> 52.7 | Improved |
| apply TAF to MiniGPT-4 | MiniGPT-4-7B · CHAIR MSCOCO | Recall: 55.8 -> 56.3 | Mixed |
| increase anchor emphasizing coefficient alpha (ablation) | TAF on LLaVA-1.5-7B · POPE | F1 | Mixed |
| raise phantom isolation ratio from 0 to 100% (ablation) | TAF on LLaVA-1.5-7B · POPE | F1 | Improved |
Closest papers by meaning
- One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination (CVPR 2026 · similarity 0.95)
- Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.94)
- Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation (AAAI 2026 · similarity 0.94)
- Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.94)
- Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens (CVPR 2025 · similarity 0.94)
- Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations (CVPR 2026 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
