Research map / Vision-language and multimodal / Multimodal LLMs
Staying VIGILant: Mitigating Visual Laziness in MLLMs via Information-Theoretic Alignment
Xi Xiao, Chen Liu, Chih-Ting Liao, Yunbei Zhang, Qizhen Lan, YUXIANG WEI, Lin Zhao, Janet Wang, Jianyang Gu, Muchao Ye, Tianyang Wang, Hao Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual anchor (counterfactual blind-state constraint) (ablation) | VIGIL on Qwen2.5-VL-7B · POPE Random | F1: 85.3 -> 88.5 | Improved |
| add visual anchor (ablation) | VIGIL on Qwen2.5-VL-7B · AMBER Generative | AMBER score: 43.1 -> 46.5 | Improved |
| add hard negatives (ablation) | VIGIL on Qwen2.5-VL-7B · POPE Random | F1: 86.5 -> 88.5 | Improved |
| add dynamic gating (ablation) | VIGIL on Qwen2.5-VL-7B · POPE Random | F1: 87.4 -> 88.5 | Improved |
| add dynamic gating (ablation) | VIGIL on Qwen2.5-VL-7B · MathVista | accuracy: 49.2 -> 49.5 | Improved |
| add visual anchor (ablation) | VIGIL on Qwen2.5-VL-7B · MathVista | accuracy: 49.0 -> 49.5 | Improved |
| replace difficulty reweighting (DA-DPO) with explicit attention-masking CVD (ablation) | DPO on Qwen2.5-VL-7B · POPE Random | F1: 85.7 -> 88.5 | Improved |
| filter easy samples (10%) instead of masking (ablation) | DPO on Qwen2.5-VL-7B · POPE Random | F1: 82.8 -> 84.3 | Improved |
| VIGIL vs DPO on high visual-dependency samples | Qwen2.5-VL-7B · POPE Random, High VDI bucket | F1: 78.2 -> 87.5 | Improved |
| VIGIL vs DPO on low visual-dependency samples | Qwen2.5-VL-7B · POPE Random, Low VDI bucket | F1: 88.5 -> 89.2 | No clear effect |
| scale base model from 7B to 72B | VIGIL · POPE Adversarial | gain over SFT (points): 4.1 -> 5.3 | Improved |
| apply VIGIL to localization (zero-shot referring expression) | Qwen2.5-VL-7B · RefCOCOg val | Acc@0.5: 44.8 -> 49.5 | Improved |
Closest papers by meaning
- On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.92)
- Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.91)
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs (NeurIPS 2025 · similarity 0.91)
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.91)
- The Alignment Illusion in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.91)
- On the Faithfulness of Visual Thinking: Measurement and Enhancement (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
