Research map / Vision-language and multimodal / Multimodal LLMs
DEGAP: Dynamic Entropy-Guided Attention Perturbation for Contrastive Decoding in Large Vision-Language Models
Hyein Seo, Yuna Jeong, Mingyu Kang, Junhyeong Park, Yong Suk Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DEGAP, a dynamic entropy-guided attention perturbation method for contrastive decoding that reduces hallucinations in large vision-language models.
Problem
LVLMs hallucinate by relying on language priors without visual grounding; existing contrastive decoding methods build contrast branches to mitigate this.
Approach
Uses entropy-guided, dynamically determined attention perturbation to construct the contrast branch for contrastive decoding, rather than fixed perturbations or input distortions (inferred from title; abstract truncated).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use entropy-guided dynamic masking layer instead of static layer 0 (ablation) | DEGAP on LLaVA-1.5 · CHAIR | CHAIRs: 25.6 -> 19.1 | Improved |
| use entropy-guided dynamic masking layer instead of static layer 5 (ablation) | DEGAP on LLaVA-1.5 · CHAIR | CHAIRs: 28.6 -> 19.1 | Improved |
| use entropy-guided dynamic masking layer instead of static layer 20 (best static) (ablation) | DEGAP on LLaVA-1.5 · CHAIR | CHAIRs: 21.0 -> 19.1 | Improved |
| use dynamic layer instead of static layer 5 (ablation) | DEGAP on InstructBLIP · CHAIR | CHAIRs: 40.6 -> 22.1 | Improved |
| use dynamic layer instead of static layer 20 (best static) (ablation) | DEGAP on mPLUG-Owl2 · CHAIR | CHAIRs: 22.2 -> 20.2 | Improved |
| increase visual token masking ratio from 20% to 100% (ablation) | DEGAP on LLaVA-1.5 · CHAIR | CHAIRs: 26.0 -> 19.1 | Improved |
| increase visual token masking ratio from 80% to 100% (ablation) | DEGAP on LLaVA-1.5 · CHAIR | CHAIRs: 22.4 -> 19.1 | Improved |
| apply DEGAP contrastive decoding over regular decoding | LLaVA-1.5 · CHAIR | CHAIRs: 27.2 -> 19.1 | Improved |
| apply DEGAP over regular decoding | InstructBLIP · POPE MS-COCO Random | accuracy: 82.60 -> 89.69 | Improved |
| apply DEGAP over regular decoding | LLaVA-1.5 · MME total score | MME score: 573.33 -> 631.11 | Improved |
| use DEGAP instead of ICD | LLaVA-1.5 · POPE Adversarial/Popular | accuracy | Mixed |
| reuse hidden state and run only later layers for the contrast branch (DEGAP vs DeGF) | DEGAP on LLaVA-1.5 · latency | latency (s): 11.86 -> 5.09 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- HALC — HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding (ICML 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Builds on: contrastive decoding
Closest papers by meaning
- Large Vision-Language Models Get Lost in Attention (ICML 2026 · similarity 0.91)
- Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models (ICML 2026 · similarity 0.91)
- Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding (CVPR 2026 · similarity 0.90)
- SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding (ICML 2025 · similarity 0.90)
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · similarity 0.90)
- ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM (AAAI 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
