Research map / Vision-language and multimodal / Multimodal LLMs
Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning
Yuyao Ge, Shenghua Liu, Yiwei Wang, Lingrui Mei, Baolong Bi, Xuanshan Zhou, Jiayu Yao, Jiafeng Guo, Xueqi Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CARVE contrastive attention masking | LLaVA-1.5-7B · V* | accuracy: 38.7 -> 66.5 | Improved |
| Apply CARVE | Qwen2.5-VL-7B · TextVQA | accuracy: 75.0 -> 81.7 | Improved |
| Apply CARVE | Qwen2.5-VL-7B · POPE | accuracy: 87.0 -> 88.6 | Improved |
| Use contrastive normalization instead of direct task attention masking (ablation) | CARVE (Qwen2.5-VL-7B) · TextVQA | accuracy: 78.6 -> 82.0 | Improved |
| Use pixel-level masking instead of in-attention modification (ablation) | CARVE (Qwen2.5-VL-7B) · TextVQA | accuracy: 77.1 -> 82.0 | Improved |
| Fuse deep layers [20,25] instead of shallow [10,15] (ablation) | CARVE (LLaVA-1.5-7B) · TextVQA | accuracy: 49.2 -> 57.9 | Improved |
| Use final-token attention instead of first-token attention (ablation) | CARVE (Qwen2.5-VL-7B) · TextVQA | accuracy: 80.7 -> 81.9 | Improved |
| Use multiplicative contrast instead of additive subtraction (ablation) | CARVE · TextVQA | relative gain: +4.53% -> +9.33% | Improved |
| Use CARVE instead of SD-RPN (trained) | Qwen2.5-VL-7B · DocVQA EM | EM: 55.90 -> 63.00 | Improved |
| Use CARVE instead of SD-RPN | Qwen2.5-VL-7B · ScienceQA | MC accuracy: 83.84 -> 82.50 | Hurt |
| Apply CARVE for knowledge reasoning (ScienceQA) | VLMs · ScienceQA | MC accuracy: +0.60% to +5.93% | Improved |
| Higher visual complexity (attention entropy 5.1 to 6.8) | Qwen2.5-VL-3B · TextVQA | accuracy: approx. 76% -> 65% | Hurt |
Closest papers by meaning
- Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs (ICLR 2026 · similarity 0.91)
- CoFFT: Chain of Foresight-Focus Thought for Visual Language Models (NeurIPS 2025 · similarity 0.91)
- From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.91)
- Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas (ICML 2025 · similarity 0.91)
- Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs (ICLR 2026 · similarity 0.91)
- Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
