Research map / Vision-language and multimodal / Multimodal LLMs

GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GroundingME, a 1,005-sample visual grounding benchmark with four challenge dimensions (discriminative, spatial, limited, rejection), evaluates 25 MLLMs, and explores test-time scaling via thinking-quality judges and rejection-focused data-mixture fine-tuning.

Problem

Existing visual grounding benchmarks are saturated and allow shortcut keyword matching, and they ignore whether MLLMs can reject ungroundable descriptions. The paper asks whether MLLMs can ground with human-like sophistication.

Approach

A multi-dimensional grounding benchmark (discriminative, spatial, limited, rejection) built from SA-1B and HR-Bench images with automatic box generation, Gemini-generated descriptions, and human refinement, including deliberately wrong descriptions for rejection; plus a best-of-16 test-time scaling by thinking-quality judges and a negative-sample data mixture for SFT.

Key results

• The best model, Qwen3-VL-235B-A22B, reaches only 45.1% total accuracy on GroundingME, and most models score 0% on the Rejection category.
• Best-of-16 test-time scaling with Qwen3-VL-A22B as a thinking-aware judge improves total accuracy from 49.8% to 54.3% (4.5% gain).
• Data-mixture SFT of Qwen3-VL-8B-Instruct at a 2:1 negative:positive ratio raises Rejection accuracy from 0% to 27.9%, while out-of-domain non-rejection performance drops from 38.8% to a max of 33.0%.

Surprising findings

• Rejection-trained gains do not generalize for free: performance on the non-rejection subset degrades after negative-sample fine-tuning.
• Commercial models such as GPT-5 and Claude-Sonnet-4.5 were excluded because their coordinate outputs were systematically distorted under the unified prompt.

Limitations

• Most open-source and commercial models score 0% on the Rejection category, and rejection gains from data mixture hurt out-of-domain grounding.
• GPT-5, Claude-Sonnet-4.5 and Grok-4 were not reported due to coordinate output issues, so the comparison is incomplete.
• Tool-use with PyVision on Claude-Sonnet-4.5 did not improve tiny-object grounding (7.3% on Small).

Experimental setup

Datasets: SA-1B HR-Bench RefCOCOg

Benchmarks: GroundingME RefCOCOg

Metrics: Accuracy@0.5 Accuracy@0.75 Accuracy@0.9 mAcc

Baselines: Qwen2.5-VL-72B GLM-4.5V Gemini-2.5-Pro Seed-1.6-Vision Llama-4-Maverick InternVL3.5-A28B

Builds on: RefCOCO Ref-L4 GroundingDINO RAM++

Closest papers by meaning

• Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.93)
• Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning (AAAI 2026 · similarity 0.92)
• PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs (ICLR 2026 · similarity 0.92)
• Grounding Multimodal Large Language Models to the World (ICLR 2024 · similarity 0.91)
• GROUNDHOG: Grounding Large Language Models to Holistic Segmentation (CVPR 2024 · similarity 0.91)
• Investigating Compositional Challenges in Vision-Language Models for Visual Grounding (CVPR 2024 · similarity 0.91)