Research map / 3D vision / Gaussian splatting

Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

NeurIPS 2026 · 3D vision › Gaussian splatting · approach: Sparse-view Gaussian splatting for novel views

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Urbex casts city-scale 3D grounding as agentic active search: a Qwen3-VL-4B VLM agent uses landmark query, zoom-in, and oblique-render tools over BEV views, trained with GRPO on CityRefer and evaluated zero-shot on STPLS3D-Refer.

Problem

City-scale 3D grounding is usually formulated as ranking precomputed candidate instances, which depends on proposal quality and scales poorly with the number of candidates in large outdoor scenes.

Approach

Expose a city-scale 3D scene as an interactive multi-view BEV environment with city-aware tools (landmark query, zoom-in, oblique rendering), so a VLM agent searches actively and commits a free-form box, trained with GRPO using smooth IoU/center-distance terminal rewards plus tool-level shaping.

Key results

• On CityRefer-ND with GT candidates, Urbex reaches 25.8 Hit@1 vs 23.4 for CityAnchor; on NO it reaches 23.9 Hit@1 vs 14.2.
• Without GT candidates on CityRefer-NO, Urbex attains 14.6 2D IoU and 7.7 3D IoU, versus 4.2/2.4 for a SoftGroup-proposal CityAnchor pipeline.
• Urbex averages about 3.2 s per query versus 32.1 s for CityAnchor, and achieves 27.2 Hit@1 zero-shot on STPLS3D-Refer.

Surprising findings

• Ablation shows that adding landmark query gives the largest tool gain (Hit@3 from 15.5 to 28.2), and a 10 m coordinate noise barely hurts accuracy.
• Thresholding Hit@1 at target IoU 0.25 drops Hit@1 by about 5 points, showing many free-form boxes are oversized despite correct identification.

Limitations

• STPLS3D-Refer is small (445 expressions) and semi-automatically generated with VLM assistance and manual filtering, and most expressions use landmarks.
• Predicted boxes are often oversized (median area ratio 1.40, P90 9.92), hurting localization quality for small targets.
• Relies on OpenStreetMap landmarks and a high-resolution BEV, and baselines were retrained under the authors' own protocol.

Experimental setup

Datasets: CityRefer SensatUrban OpenStreetMap STPLS3D

Benchmarks: CityRefer STPLS3D-Refer

Metrics: Hit@1 Hit@3 Hit@5 2D IoU 3D IoU point F1 inference time

Baselines: GPT-5.4 Qwen3.6-plus InstanceRefer EDA CityRefer CityAnchor

Builds on: GRPO CityRefer CityAnchor Qwen3-VL

Closest papers by meaning

• CitySTAR: Agent-Driven Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Grounding (NeurIPS 2026 · similarity 0.87)
• SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding (NeurIPS 2025 · similarity 0.86)
• DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.86)
• 3EED: Ground Everything Everywhere in 3D (NeurIPS 2025 · similarity 0.86)
• From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.85)
• Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.85)