Research map / Vision-language and multimodal / CLIP and zero-shot
SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SPAZER is a training-free VLM agent that combines multi-view 3D renderings for spatial candidate screening with retrieved 2D camera views for semantic verification to perform zero-shot 3D visual grounding.
Problem
Zero-shot 3D visual grounding requires both spatial reasoning over 3D scene layout and fine-grained semantic verification, but existing zero-shot methods emphasize either 3D-based or 2D-based understanding alone.
Approach
Progressively decouple reasoning: a VLM selects an optimal holistic 3D rendering view and filters candidate objects via retrieval-augmented anchor filtering, then uses 2D camera views mapped from 3D anchor boxes for joint 3D-2D decision-making.
Key results
- On ScanRefer, SPAZER reaches 57.2% Acc@0.25 and 48.8% Acc@0.5 overall, versus 51.6/32.8 for VLM-Grounder and 49.6/39.8 for CSVG.
- On Nr3D without GT object classes, SPAZER achieves 63.8% overall accuracy, above zero-shot SeeGround (46.1) and VLM-Grounder (48.0).
- With GT object classes on Nr3D, SPAZER reaches 73.2% overall, surpassing Transcrib3D (70.2) and CSVG (59.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SPAZER's spatial-semantic progressive reasoning instead of VLM-Grounder | Zero-shot 3D visual grounding · ScanRefer Overall | Acc@0.25: 51.6 -> 57.2; +24.4% | Helped |
| Use SPAZER instead of CSVG | Zero-shot 3D visual grounding · ScanRefer Overall | Acc@0.5: 39.8 -> 48.8; +8.9% | Helped |
| Use SPAZER instead of EaSe | Zero-shot 3D visual grounding · Nr3D Overall (no GT object class) | accuracy: 52.9 -> 63.8 | Helped |
| Use holistic view selection instead of BEV (ablation) | SPAZER · Nr3D Overall | accuracy: +6.3% | Helped |
| Use holistic view selection instead of a random view (ablation) | SPAZER · Nr3D Overall | accuracy: +3.6% | Helped |
| Add retrieval-augmented anchor filtering to text-based matching (ablation) | SPAZER candidate screening · Nr3D | accuracy: 2% to 7% | Helped |
| Add 2D camera views for 3D-2D joint decision-making (ablation) | SPAZER · Nr3D | accuracy: 11.6% | Helped |
| Increase Top-k from 4 to 5 (ablation) | SPAZER candidate screening · Nr3D | overall accuracy | Hurt |
| Increase rendered views from n=0 to n=4 (ablation) | SPAZER holistic view selection · Nr3D | accuracy | Helped |
| Increase rendered views from n=4 to n=8 (ablation) | SPAZER holistic view selection · Nr3D | accuracy | Hurt |
| Replace GPT-4o with GPT-4.1 (ablation) | SPAZER · Nr3D Overall | accuracy: 63.8 -> 64.3 | Helped |
| Replace GPT-4o with GPT-4o-mini (ablation) | SPAZER · Nr3D Overall | accuracy: 63.8 -> 46.4 | Hurt |
Surprising findings
- Switching the VLM from GPT-4o to GPT-4.1 raises Nr3D overall accuracy only slightly (63.8 to 64.3), while open-source Qwen2.5-VL-72B reaches 56.0 on the subset.
Limitations
- Grounding accuracy depends on the pre-trained 3D detector (Mask3D) for bounding boxes.
- 3D-to-camera mapping is affected by inaccurate camera parameters and depth.
- Evaluation uses a 250-sample subset of each dataset to save budget.
Experimental setup
Datasets: ScanRefer Nr3D ScanNet
Benchmarks: ScanRefer Nr3D
Metrics: Acc@0.25 Acc@0.5 accuracy
Baselines: ScanRefer 3DVG-Transformer BUTD-DETR ChatScene Video-3D LLM GPT4Scene OpenScene LLM-Grounder ZSVG3D VLM-Grounder SeeGround CSVG
Builds on: VLM-Grounder SeeGround ZSVG3D CSVG
Closest papers by meaning
- S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.91)
- From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.90)
- Chain of Semantics Programming in 3D Gaussian Splatting Representation for 3D Vision Grounding (CVPR 2025 · similarity 0.90)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.90)
- SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding (CVPR 2025 · similarity 0.90)
- Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning (AAAI 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.