Research map / 3D vision / Depth and stereo
Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes DUSt3R to predict structural-plane pointmaps from unposed sparse indoor views, then merges per-view planes into a multi-view room layout with 2D detection and a min-cut-style merging step.
Problem
Room layout estimation from multiple unposed, sparse perspective images, which traditionally requires multi-step SfM (camera calibration, matching, triangulation) and is poorly studied due to scarce multi-view layout data.
Approach
Retrain DUSt3R with occlusion-free structural-plane depth targets (from Structure3D plane annotations) so it outputs plane pointmaps directly, then combine them with a Noncuboid 2D plane detector and a post-processing merge that aligns walls in a floor-plan projection and merges duplicates via a min-cut formulation.
Key results
- Plane-DUSt3R (aligned) reaches 76.84% re-IoU and 52.63% 3D precision on Structure3D vs 74.51% and 37.00% for Noncuboid + MASt3R.
- On Structure3D pairwise pose, Plane-DUSt3R (aligned) gets mAA@30 of 91.80 vs 85.34 for MASt3R.
- On CAD-Estate subset, Plane-DUSt3R (metric) gets re-IoU 63.14% vs 55.99% for Noncuboid + GT pose.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Noncuboid single-view plane normals with Plane-DUSt3R predictions in the aligned pipeline | Multi-view room layout estimation · Structure3D test | 3D precision: 46.96 -> 52.63 | Helped |
| Use aligned rather than metric-scale Plane-DUSt3R (ablation) | Multi-view room layout estimation · Structure3D test | 3D precision: 48.98 -> 52.63 | Helped |
| Fine-tune DUSt3R to predict structural planes | DUSt3R · Structure3D | mAA@30: 85.34 -> 91.80 | Helped |
| Retrain Noncuboid with hyperparameters selected by grid search | Noncuboid 2D plane detector · Structure3D | IoU: 79.94 -> 80.18 | Helped |
| Increase the number of input views from 2 to 5 (ablation) | Multi-view room layout estimation · Structure3D test | 3D precision: 53.19 -> 55.02 | Helped |
| Increase the number of input views from 2 to 5 (ablation) | Multi-view room layout estimation · Structure3D test | 3D recall: 42.60 -> 49.59 | Helped |
| Use metric-scale Plane-DUSt3R instead of Noncuboid with ground-truth poses | Multi-view room layout estimation · CAD-Estate | re-IoU: 55.99 -> 63.14 | Helped |
| Use metric-scale Plane-DUSt3R instead of Noncuboid with ground-truth poses | Multi-view room layout estimation · CAD-Estate | re-PE: 20.33 -> 15.15 | Helped |
Surprising findings
- Plane-DUSt3R outperforms MASt3R on pose-related metrics even though the paper notes it is fine-tuned on Structure3D, which may favor it in-domain.
Limitations
- Assumes a single floor, single ceiling, and walls perpendicular to floor and ceiling (Manhattan-style).
- Trained and primarily evaluated on synthetic Structure3D; RealEstate10K and in-the-wild results are qualitative.
- Pose-related gains are reported mainly against MASt3R on a fine-tuned domain.
Experimental setup
Datasets: Structured3D RealEstate10K CAD-Estate
Benchmarks: Structured3D RealEstate10K CAD-Estate
Metrics: IoU PE EE RMSE RRA@15 RTA@15 mAA@30 3D precision 3D recall
Baselines: Noncuboid + MASt3R Noncuboid + GT pose DUSt3R MASt3R Planar R-CNN Rac
Builds on: DUSt3R MASt3R Noncuboid
Closest papers by meaning
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.90)
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.89)
- HouseLayout3D: A Benchmark and Training-free Baseline for 3D Layout Estimation in the Wild (NeurIPS 2025 · similarity 0.88)
- MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.88)
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.87)
- MET3R: Measuring Multi-View Consistency in Generated Images (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.