Research map / 3D vision / 3D generation
3D Aware Region Prompted Vision Language Model
An-Chieh Cheng, Yang Fu, Yukang Chen, Zhijian Liu, Xiaolong Li, Subhashree Radhakrishnan, Song Han, Yao Lu, Jan Kautz, Pavlo Molchanov, Hongxu Yin, Xiaolong Wang and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add single-view 2D pre-training before multi-view fine-tuning (ablation) | SR-3D (3D PE) · ScanQA val | CIDEr: 102.9 -> 109.3 | Improved |
| add single-view 2D pre-training before multi-view fine-tuning (ablation) | SR-3D (3D PE) · SQA3D test | exact match: 59.1 -> 62.2 | Improved |
| add 3D positional embeddings without 2D pre-training (ablation) | SR-3D 3D fine-tuned · ScanQA val | CIDEr: 101.3 -> 102.9 | No clear effect |
| add 3D positional embeddings on top of 2D pre-training (ablation) | SR-3D 3D fine-tuned · ScanQA val | CIDEr: 108.2 -> 109.3 | Improved |
| add 2D pre-training (ablation) | SR-3D · SR-3D-Bench global Distance | success rate: 60.3 -> 74.8 | Improved |
| add 2D pre-training (ablation) | SR-3D · SR-3D-Bench Multi. Complex | accuracy: 59.0 -> 76.0 | Improved |
| add 3D positional embeddings and region extractor to NVILA-Lite-8B | SR-3D-8B · RealWorldQA | accuracy: 65.6 -> 68.1 | Improved |
| add 3D positional embeddings and region extractor to NVILA-Lite-8B | SR-3D-8B · OCR benchmarks (TextVQA, ChartQA, DocVQA) | accuracy: TextVQA 78.1 -> 77.3; DocVQA 91.7 -> 91.0 | Hurt |
| add 3D positional embeddings and region extractor to NVILA-Lite-8B | SR-3D-8B · General knowledge (GQA, AI2D, POPE) | accuracy: GQA 65.3 -> 64.2 | No clear effect |
| replace ground-truth point clouds with Cut3R-estimated point maps at test time | SR-3D · ScanQA val | CIDEr: 109.3 -> 109.3 | No clear effect |
| replace ground-truth point clouds with Cut3R-estimated point maps at test time | Video-3D LLM · ScanQA val | CIDEr: 102.1 -> 100.7 | Hurt |
| add region prompts and 3D positional embeddings (versus SoM-prompted VLMs) | SR-3D-8B · SR-3D-Bench qualitative | accuracy: 47.1 -> 79.5 | Improved |
Closest papers by meaning
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.91)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
- GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models (ICLR 2026 · similarity 0.91)
- Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.90)
- Surprise3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes (NeurIPS 2025 · similarity 0.90)
- RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
