Research map / Vision-language and multimodal / Multimodal LLMs
Decoding-time hallucination mitigation
Alters decoding, such as contrastive or layer-wise decoding, to reduce object hallucination in vision-language models.
91 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2024–2026) · an approach in Multimodal LLMs, keyword cluster “hallucination · lvlms · mitigating”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
12 papers in 2023–24 and 79 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 5.8% to 3.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2024 · Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · 172 citations)
- 2025 · Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · 20 citations)
- 2026 · ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM (AAAI 2026 · 2 citations)
Earliest papers here
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · 172 citations)
- HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models (CVPR 2024 · 138 citations)
- OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation (CVPR 2024 · 124 citations)
What has been tried
37 papers here report 446 measured changes (a component added, removed or swapped, compared with a baseline): helped 72%, no clear effect 4%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use T=999 instead of T=500 noise steps | MME hallucination subset · Total Scores | 591.67 (± 36.06) -> 557.78 (± 1.92) | Hurt | Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · 172 citations) |
| Add over-trust penalty only (no retrospection) | CHAIR 512 tokens · CHAIR_S | Shikra 50.4 -> 41.8; LLaVA-1.5 48.8 -> 45.2 | Helped | OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation (CVPR 2024 · 124 citations) |
| add contrastive learning without hallucinative captions | MMHal-Bench · overall score | 1.55 -> 1.67 | Helped | Hallucination Augmented Contrastive Learning for Multimodal Large Language Model (CVPR 2024 · 79 citations) |
| set λ to 0.001 instead of 0.02 | MS COCO val captioning · CHAIRi | 5.8 -> 7.3 | Hurt | Multi-Modal Hallucination Control by Visual Information Grounding (CVPR 2024 · 66 citations) |
| Finetune MiniGPT4 on LRV-Instruction | MME Perception and Cognition · score | Perception 616.41 -> 895.96; Cognition 232.71 -> 296.43 | Helped | Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning (ICLR 2024 · 23 citations) |
| add hallucination severity weighting to DPO | Object HalBench · CHAIR_S | 6.7 -> 5.3 | Helped | Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · 20 citations) |
| Evaluate using the POPE image and class subsampling strategy instead of all classes | COCO · F^0.5_CLS | 12.3 points | Hurt | THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models (CVPR 2024 · 18 citations) |
| use a 120M-parameter matching model instead of the standard matching model | POPE · F1 Score | 84.58 -> 84.11 | Hurt | Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention (CVPR 2025 · 14 citations) |
| Use FarSight positional awareness encoding instead of RoPE | CHAIR_S · CHAIR_S | 48.0 -> 41.6 | Helped | Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding (CVPR 2025 · 10 citations) |
| Edit layers 24-32 rather than 16-32 | MSCOCO val, CHAIR · CHAIRS | 15.2 -> 16.2 | Hurt | Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection (CVPR 2025 · 9 citations) |
| add adaptive information constraint (dynamic β) | MMVP · overall score | 28.67 -> 32.00; 32.67 -> 41.33; 45.33 -> 48.00 | Helped | CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models (NeurIPS 2024 · 6 citations) |
| add adaptive beta | MSCOCO · CHAIR_S | 17.1 -> 16.2; 19.2 -> 18.4 | Helped | Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow (AAAI 2025 · 5 citations) |
What reviewers will expect
Datasets and baselines named in the 79 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MSCOCO · 27 of 79 papers
- POPE · 22 of 79 papers
- CHAIR · 14 of 79 papers
- GQA · 11 of 79 papers
- MME · 10 of 79 papers
- AMBER · 8 of 79 papers
- A-OKVQA · 8 of 79 papers
- Visual Genome · 6 of 79 papers
Baselines
- VCD · 48 of 79 papers
- OPERA · 35 of 79 papers
- DoLa · 17 of 79 papers
- Beam Search · 13 of 79 papers
- PAI · 11 of 79 papers
- ICD · 10 of 79 papers
- AGLA · 10 of 79 papers
- HALC · 8 of 79 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 91 papers; from OpenAlex and the papers' affiliations).
- Alibaba Group · 4 papers
- University of Maryland, College Park · 3 papers
- Nanyang Technological University · 3 papers
- Amazon · 3 papers
- Seoul National University · 3 papers
- Hong Kong University of Science and Technology · 3 papers
- Xi’an Jiaotong University · 3 papers
- Zhejiang University · 2 papers
- Ant Group · 2 papers
- Tsinghua University · 2 papers
Use it
Download BibTeX of 20 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Preference tuning against hallucination (58 papers)
- Bias-aware multi-image robustness (44 papers)
Elsewhere in the topic
- Chain-of-thought multimodal reasoning (332 papers)
- VLM capability analysis (264 papers)
- Multimodal LLM benchmarks (252 papers)
- Spatial reasoning in VLMs (207 papers)
- Instruction-tuned multimodal assistants (199 papers)
- Benchmarks and analysis of MLLMs (181 papers)
- Reinforcement learning for visual reasoning (163 papers)
- Interleaved image-text generation with MLLMs (153 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).