Research map / 3D vision / 3D generation
Rooms from Motion: Un-posed Indoor 3D Object Detection as Localization and Mapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Rooms from Motion replaces the keypoint matcher of structure-from-motion with an object-centric matcher over image-derived 3D oriented boxes, estimating metric camera poses and building a semantic 3D object map from un-posed RGB(-D) images.
Problem
Scene-level 3D object detection and camera localization typically rely on explicit point clouds or dense volumes and on a priori metric camera poses; the paper asks whether objects alone can serve as the primitive for localization and mapping from un-posed images.
Approach
Object-centric SfM: detect metric 3D oriented boxes per image with Cubify Transformer, match objects across views with a LightGlue-style object matcher (plus a corner-level matcher), estimate relative poses from matched boxes via Kabsch with 4-DoF, average poses with glomap-style averaging, form object tracks, and optionally refine boxes with bundle adjustment on box corners.
Key results
- On CA-1M with GT depth and poses, Rooms from Motion (Posed) reaches 47.4 AP15 vs 28.7 for posed FCAF.
- Under monocular RGB with GT poses, RfM (Posed + BA) achieves 31.3 AP15 on CA-1M vs 18.6 for ImGeoNet (posed).
- On ScanNet++ RGB-D localization, RfM achieves 1.1 deg median ARE and 3.5 cm ATE vs 62.8 deg and 110.1 cm for DROID-SLAM (250 frames).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use estimated un-posed camera poses rather than provided poses | Rooms from Motion · CA-1M, RGB-D mapping | AP15: 47.4 -> 39.6 | Hurt |
| use estimated un-posed camera poses rather than provided poses | Rooms from Motion · ScanNet++, RGB-D mapping | AP15: 38.8 -> 33.4 | Hurt |
| use estimated un-posed camera poses rather than provided poses | Rooms from Motion · CA-1M, RGB mapping | AP15: 24.6 -> 17.0 | Hurt |
| use estimated un-posed camera poses rather than provided poses | Rooms from Motion · ScanNet++, RGB mapping | AP15: 27.5 -> 18.6 | Hurt |
| add partial bundle adjustment (ablation) | Rooms from Motion (Posed) · CA-1M, RGB mapping | AP15: 24.6 -> 31.3 | Helped |
| add partial bundle adjustment (ablation) | Rooms from Motion (Posed) · ScanNet++, RGB mapping | AP15: 27.5 -> 30.5 | Helped |
| expand ScanNet++ taxonomy to include all available objects as unlabeled (ablation) | Rooms from Motion · ScanNet++ localization | Registered: 93% -> 97% | Helped |
| use the coarse ARKitScenes taxonomy instead of exhaustive CA-1M object labels (ablation) | Rooms from Motion · CA-1M localization | Registered: 93% -> 61% | Hurt |
| use standard rather than high field-of-view images (ablation) | Rooms from Motion · ScanNet++ localization | ARE (deg): 2.4 -> 6.1 | Hurt |
| use standard rather than ultrawide images (ablation) | Rooms from Motion · CA-1M localization | ARE (deg): 2.5 -> 7.9 | Hurt |
| use RGB instead of RGB-D input | Rooms from Motion · CA-1M localization | ATE (cm): 4.0 -> 12.7 | Hurt |
| use RGB instead of RGB-D input | Rooms from Motion · ScanNet++ localization | ATE (cm): 3.5 -> 15.3 | Hurt |
Surprising findings
- Rooms from Motion on RGB only is competitive with RGB-D DROID-SLAM running on 250 frames on CA-1M.
- Localization is not sensitive to the actual class labels of objects; class-agnostic or 'unlabeled' objects work nearly as well.
Limitations
- Limited to indoor scenes due to lack of diverse training data.
- Only images containing objects can be registered.
- Relocalization and map updating with new images remain limited (further work needed).
Experimental setup
Datasets: CA-1M ScanNet++ ARKitScenes
Benchmarks: CA-1M ScanNet++
Metrics: AP15 AR15 AP25 AR25 ARE ATE registered percentage
Baselines: FCAF UniDet3D ImVoxelNet ImGeoNet DROID-SLAM CUT3R
Builds on: Cubify Transformer LightGlue glomap SLAM++
Closest papers by meaning
- Cubify Anything: Scaling Indoor 3D Object Detection (CVPR 2025 · similarity 0.89)
- UniDet3D: Multi-dataset Indoor 3D Object Detection (AAAI 2025 · similarity 0.88)
- SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection (AAAI 2024 · similarity 0.87)
- LOCATE 3D: Real-World Object Localization via Self-Supervised Learning in 3D (ICML 2025 · similarity 0.87)
- State Space Model Meets Transformer: A New Paradigm for 3D Object Detection (ICLR 2025 · similarity 0.87)
- Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.