Research map / Vision-language and multimodal / Multimodal LLMs
Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenes
Yuling Xi, Haokai Zhang, Muzhi Zhu, Hao Zhong, Zongze Du, Hengyu Zhao, Chenchen Jing, Yufei Yin, Bin Qin, Yongjie Yang, Zhenbo Luo, Hao Chen and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reference metrics (N_ref 0 -> 4) to the prompt (ablation) | MetricReasoner · Metric-Bench | MRA: 47.56 -> 48.59; RMSE 0.5540 -> 0.5323 | Improved |
| increase references from 4 to 5 (ablation) | MetricReasoner · Metric-Bench | MRA: 48.59 -> 46.85 | Hurt |
| add EP and BP accuracy rewards versus none (ablation) | MetricReasoner · Metric-Bench | MRA: 12.38 -> 48.59 | Improved |
| add BP reward on top of EP (ablation) | MetricReasoner · Metric-Bench | MRA: 45.95 -> 48.59 | Improved |
| add EP reward on top of BP (ablation) | MetricReasoner · Metric-Bench | MRA: 48.05 -> 48.59 | Improved |
| use finer BP bins (5 intervals instead of 4) (ablation) | MetricReasoner · Metric-Bench | MRA: 48.59 -> 46.14 | Hurt |
| use finer BP bins (6 intervals instead of 4) (ablation) | MetricReasoner · Metric-Bench | MRA: 48.59 -> 48.88 | No clear effect |
| use finer BP bins (8 intervals instead of 4) (ablation) | MetricReasoner · Metric-Bench | MRA: 48.59 -> 47.69 | Hurt |
| train with RL instead of SFT (ablation) | Qwen3-VL-8B · BLINK | accuracy: 51.26% -> 81.82% | Improved |
| train with RL instead of SFT (ablation) | Qwen3-VL-8B · RoboSpatial | accuracy: 40.86 -> 55.14 | Improved |
| add CoT SFT stage before RL (ablation) | MetricReasoner · BLINK | accuracy: 81.82% -> 34.53% | Hurt |
| metric-guided RFT versus base model | Qwen3-VL-8B · V* Bench | accuracy: 61.78% -> 68.59% | Improved |
Closest papers by meaning
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.90)
- InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
- Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.89)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.88)
- QUANTIPHY: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
