AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

CofactGVR: Counterfactual Intervention for Grounded Visual Reasoning

Yan Zhang, Zhijin Qin, Feng Xu, Guiguang Ding, Jungong Han

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CofactGVR, which decomposes grounded visual reasoning into evidence generation followed by counterfactual answer reasoning to provide learning signals for intermediate grounding and reduce shortcut solutions.

Problem

Grounded visual reasoning (GVR) with MLLMs and RL-style fine-tuning lacks effective learning signals for intermediate grounding decisions and is prone to shortcut solutions.

Approach

Explicitly decomposes GVR into Evidence Generation followed by Counterfactual Answer Reasoning, formalized as a Causal Grounding Graph (CGG) in which the generated evidence causally influences the answer; counterfactual interventions on the evidence provide training signals.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add counterfactual intervention bonus (ablation)CofactGVR · MMEscore: 2349.1 -> 2428.5Improved
add Q-PAS (quantile filter plus prioritized sampling) (ablation)CofactGVR · MMEscore: 2406.2 -> 2428.5Improved
bound the counterfactual bonus with floor and cap (ablation)CofactGVR · MMEscore: 2341.6 -> 2428.5Improved
gate the bonus to high-quality factual rollouts (ablation)CofactGVR · RealworldQAaccuracy: 61.7 -> 66.9Improved
combine correctness mask and top-k fallback instead of either alone (ablation)CofactGVR · MMEscore: 2302.5 (top-k only) -> 2428.5Improved
use prioritization exponent alpha 0.5 instead of 0 (ablation)Q-PAS · MMEscore: 2372.7 -> 2428.5Improved
raise prioritization exponent to 1.0 or above (ablation)Q-PAS · MMEscore: 2428.5 -> 2389.3Hurt
add filtering before resampling in Q-PAS (ablation)Q-PAS · RealworldQAaccuracy: 63.8 -> 66.9Improved
use noise-corrupted random crop instead of no-evidence or plausible-region counterfactual (ablation)CofactGVR · MMEscore: 2306.0 / 2386.4 -> 2428.5Improved
train with CofactGVR instead of Ground-R1Qwen-based MLLM · RefCOCOg valaccuracy: 90.1 -> 89.0Hurt
train with CofactGVR instead of Ground-R1Qwen-based MLLM · RefCOCO+ test-Baccuracy: 78.8 -> 85.3Improved
preserve predicted evidence versus randomize it at inferenceCofactGVR vs Ground-R1 · MMEscore gap: 78.7 vs -22.1Improved

Experimental setup

Builds on: GRPO

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy