Research map / Vision-language and multimodal / Multimodal LLMs

GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GeM-VG, a Qwen2-VL-7B-based MLLM for generalized multi-image visual grounding (any number of targets), trained with SFT on a new MG-Data-240K set followed by a hybrid CoT/direct-answer GRPO with multi-dimensional rewards and reward modulation.

Problem

Existing multi-image grounding models are limited to single-target localization and a narrow set of grounding task types, lacking unified modeling for generalized grounding across varying target counts and cross-image relations.

Approach

A taxonomy of multi-image grounding tasks (referential retrieval, cross-image association, reasoning), a multi-target dataset, and an R1-style GRPO with precision/recall/image/format rewards plus a reward-modulated hybrid of CoT and direct-answer modes.

Key results

• On MIG-Bench, GeM-VG reaches 74.65 average, above UniVG-R1 (72.64), a reported 2.0% gain.
• On MC-Bench, GeM-VG achieves 32.8 AP50 versus Migician's 23.1 and Qwen2-VL-7B's 20.2.
• On ODINW, GeM-VG scores 41.1 versus UniVG-R1's 33.3 and Qwen2-VL-7B's 32.0.

Surprising findings

• Pure direct-answer SFT outperforms CoT on perception-heavy subtasks such as Region grounding, while CoT is better on reasoning subtasks.
• Without reward modulation, RL quickly collapses to direct-answer mode.

Limitations

• Performance on MIG-Bench spatial/association subtasks (e.g., Robust, Common, Co-Re) is below UniVG-R1 in some cases.
• Mixed results on some ODINW subsets (e.g., AerialDrone, pothole) relative to the base model.

Experimental setup

Datasets: MG-Data-240K (self-constructed) MGrounding-630k M4-Instruct GRefCOCO ODINW RefCOCO UniVG-R1-SFTData UniVG-R1-RLData D^3 COCO Ego-Exo4D MVTrack

Benchmarks: MIG-Bench MC-Bench ODINW ODINW-13 LLMSeg ReasonVOS ReVOS MuirBench MMIU MIBench BLINK

Metrics: Acc@0.5 AP50 accuracy

Baselines: Migician UniVG-R1 Qwen2-VL-7B Qwen2.5-VL-7B Gemini-1.5 Pro GPT-4o InternVL2-8B mPLUG-Owl3 Mantis LLaVA-OV-7B MiniCPM-V 2.6

Builds on: GRPO DeepSeek-R1 Qwen2-VL Migician UniVG-R1 Open-R1

Closest papers by meaning

• SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusion (NeurIPS 2024 · similarity 0.92)
• Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning (AAAI 2026 · similarity 0.92)
• GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation (CVPR 2026 · similarity 0.91)
• PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs (ICLR 2026 · similarity 0.91)
• DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Models (CVPR 2026 · similarity 0.91)
• Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.91)