Research map / 3D vision / 3D generation
Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Physical AI Smart Spaces (PAISS), a large synthetic-plus-real benchmark of 282.5 hours of multi-camera 1080p video with 3D/2D annotations, Cosmos Transfer appearance augmentation, a hidden real-world Sim2Real target, and a 3D HOTA evaluation protocol run via AI City Challenge.
Problem
Multi-camera 3D perception and tracking in indoor smart spaces lacks large-scale, multi-class, geometry-rich benchmarks with calibration, depth, and controlled synthetic-to-real evaluation.
Approach
A benchmark combining Isaac Sim synthetic generation with automatic 2D/3D labels, Cosmos Transfer 2.5 appearance augmentation that preserves labels, a hidden real-world warehouse target calibrated with VGGT, and a 3D instantiation of HOTA (location- and box-based) served through a server-side evaluation system.
Key results
- Top 2024 AI City Challenge Track 1 team (Met_BEV, MCBLT) reached 81.2 HOTA on people tracking with 3D locations.
- Top 2025 Track 1 team (ZV) reached 69.9 raw HOTA on multi-class 3D box tracking; Metropolis-Sparse4D reached 67.3 as the only RGB-only online method.
- Full release spans 139 scenes, 1,799 cameras, 282.5 hours of 1080p video, over 72M 3D labels and 241M 2D boxes.
Surprising findings
- The only RGB-only online method in 2025 (Metropolis-Sparse4D) achieved HOTA close to top depth-using methods.
- Several 2025 RGB+depth systems scored far lower (e.g., TeamQDT 31.6, UTE AI Lab 28.0), showing wide spread among entries.
Limitations
- Data is dominated by warehouse environments and static, pre-calibrated indoor cameras.
- Synthetic avatars come from a finite asset library unbalanced across body type, skin tone, age, and mobility, so not suitable for fairness studies.
- Automatic labels and calibration are assumed accurate enough to serve as supervision; the real-world target is held out and small (4 and 7 cameras).
Experimental setup
Datasets: PAISS 2024 PAISS 2025 PAISS 2026
Benchmarks: PAISS 2024 leaderboard PAISS 2025 leaderboard PAISS 2026 hidden real-world target
Metrics: HOTA 3D HOTA DetA AssA
Baselines: MCBLT Yachiyo ZV Metropolis-Sparse4D SKKU-AutoLab (DepthTrack) TeamQDT UTE AI Lab (VGCRTrack)
Builds on: HOTA TrackEval VGGT Cosmos Transfer 2.5 Isaac Sim
Closest papers by meaning
- Rooms from Motion: Un-posed Indoor 3D Object Detection as Localization and Mapping (NeurIPS 2025 · similarity 0.87)
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.86)
- InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts (NeurIPS 2025 · similarity 0.85)
- RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigation (CVPR 2025 · similarity 0.84)
- GenSpace: Benchmarking Spatially-Aware Image Generation (NeurIPS 2025 · similarity 0.84)
- AmaraSpatial-10K: A Spatially and Semantically Aligned 3D Dataset for Spatial Computing and Embodied AI (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.