Research map / Vision-language and multimodal / Multimodal LLMs
Grasp Any Region: Prompting MLLM to Understand the Dense World
Haochen Wang, Yuhao Wang, Tao Zhang, Yikang Zhou, Yanwei Li, Jiacong Wang, Jiani zheng, Ye Tian, Jiahao Meng, Zilong Huang, Guangcan Mai, Anran Wang and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Grasp Any Region (GAR), a region-level MLLM that models given regions together with global context to understand dense scenes; details beyond the abstract are not available.
Problem
Region-level MLLMs understand given regions in isolation and neglect global context, which hurts fine-grained understanding of dense scenes with intricate details and object inter-relationships.
Approach
Prompts the MLLM with regions while explicitly incorporating global context, so region understanding accounts for object inter-relationships rather than isolated regions. Full method details are not available from the abstract provided.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RoI-aligned feature replay instead of local-only image (ablation) | GAR-1B (PerceptionLM-1B) · GAR-Bench-Cap | accuracy: 20.1 -> 57.5 | Improved |
| replace DAM-like gated cross-attention with RoI-aligned feature replay (ablation) | GAR-1B · GAR-Bench-VQA | accuracy: 40.0 -> 50.6 | Improved |
| replace global+cropped local image with RoI-aligned feature replay (ablation) | GAR-1B · GAR-Bench-Cap | accuracy: 28.4 -> 57.5 | Improved |
| replace global+cropped local image with RoI replay (ablation) | GAR-1B · DLC-Bench (positive) | accuracy: 70.1 -> 66.2 | Hurt |
| RoI replay across base models (ablation) | GAR · GAR-Bench-Cap (Qwen2.5-VL-3B) | accuracy: 34.3 -> 41.2 | Improved |
| RoI replay across base models (ablation) | GAR · GAR-Bench-Cap (InternVL3-2B) | accuracy: 32.8 -> 43.1 | Improved |
| add Fine-Grained Dataset-456K to seed data (ablation) | GAR-1B · DLC-Bench | accuracy: 74.4 -> 77.5 | Improved |
| add Relation Dataset-414K (ablation) | GAR-1B · GAR-Bench-Cap | accuracy: 14.2 -> 57.5 | Improved |
| add Relation Dataset-414K (ablation) | GAR-1B · DLC-Bench | accuracy: 77.5 -> 77.1 | No clear effect |
| fine-tune PerceptionLM-8B into GAR-8B for region understanding | PerceptionLM-8B · V* (general) | accuracy: 69.1 -> 59.2 | Hurt |
| add 600K general VQA data to GAR-8B | GAR-8B · MMStar | accuracy: 43.9 -> 51.6 | Improved |
| use GAR-8B zero-shot vs in-domain VideoRefer-7B | GAR-8B · VideoRefer-Bench-Q | accuracy: 71.9 -> 72.0 | Improved |
Closest papers by meaning
- RegionGPT: Towards Region Understanding Vision Language Model (CVPR 2024 · similarity 0.89)
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.89)
- VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
- Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.88)
- Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models (NeurIPS 2025 · similarity 0.88)
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
