Research map / Vision-language and multimodal / Multimodal LLMs

MonoVLM: Monocular 3D Visual Grounding with Vision Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MonoVLM, a three-stage GRPO curriculum (2D grounding, 3D center prediction, full 3D box with composite rewards) that turns 7B VLMs into monocular 3D visual grounding models that beat zero-shot VLMs and some vision-only specialists on Mono3DRefer.

Problem

Current VLMs, including GPT-5, perform near-zero on monocular 3D visual grounding (predicting a 3D box from an image and text description), while specialized vision-only models lack semantic language understanding.

Approach

A coarse-to-fine sequence of GRPO stages: first optimize 2D IoU for localization, then 3D center distance reward (leveraging unprojection), then full 3D IoU combined with component rewards for center, size, and yaw.

Key results

• MonoVLM-MiMo achieves 69.41 Acc@0.25 and 42.96 Acc@0.5 overall on Mono3DRefer, exceeding Mono3DVG-TGE's 68.44/51.21 only on Acc@0.25 and surpassing Cube R-CNN.
• MonoVLM-MiMo overall mIoU 38.11 versus 7.53 for the best zero-shot VLM baseline (GPT-5), over 5x improvement.
• Staged training improves MonoVLM-Qwen mIoU monotonically: 19.81 (Stage 1) to 20.89 (Stage 2) to 29.13 (Stage 3).

Surprising findings

• A naive GRPO run with only 3D IoU reward gives accurate depth but large lateral and vertical center errors, indicating the sparse IoU reward fails to teach 2D grounding.
• Stage 2 center-distance reward also improved 2D IoU as a collateral effect, despite no 2D supervision.

Limitations

• Evaluated only on Mono3DRefer; generalization to other 3D grounding benchmarks not shown.
• MonoVLM-Qwen does not beat Mono3DVG-TGE on Acc@0.25 overall (61.89 vs 68.44) and remains below on several splits.
• Requires camera intrinsics at inference time.

Experimental setup

Datasets: Mono3DRefer

Benchmarks: Mono3DRefer

Metrics: Acc@0.25 Acc@0.5 mIoU

Baselines: GPT-5 GPT-o3 Gemini-2.5-Pro Gemini-2.5-Flash Qwen2.5-VL-72B MiMo-VL-7B Mono3DVG Mono3DVG-TGE Cube R-CNN ZSGNet FAOA ReSC

Builds on: GRPO Qwen2.5-VL MiMo-VL EasyR1 Mono3DVG

Closest papers by meaning

• Mono3DVG: 3D Visual Grounding in Monocular Images (AAAI 2024 · similarity 0.91)
• Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.91)
• Beyond Human Perception: Understanding Multi-Object World from Monocular View (CVPR 2025 · similarity 0.90)
• From Thousands to Billions: 3D Visual Language Grounding via Render-Supervised Distillation from 2D VLMs (ICML 2025 · similarity 0.89)
• SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding (CVPR 2025 · similarity 0.89)
• DepthLM: Metric Depth from Vision Language Models (ICLR 2026 · similarity 0.89)