Research map / 3D vision / Depth and stereo
PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Plana3R predicts sparse planar 3D primitives and metric relative pose from unposed stereo pairs with a ViT, trained via differentiable planar splatting using only depth and normal supervision.
Problem
Metric, pose-free, zero-shot 3D reconstruction of indoor scenes from two views, avoiding dense 3D plane annotations and accurate multi-view poses required by prior planar reconstruction methods.
Approach
Replace dense point maps with sparse planar primitives predicted by a DUSt3R-style transformer, supervised by rendering depth and normal maps through PlanarSplatting's differentiable rasterizer, plus a hierarchical low/high-resolution primitive scheme gated by normal gradients.
Key results
- On ScanNetV2 two-view reconstruction, Plana3R reports Chamfer 0.11 and F-score 92.52 versus MASt3R's 0.21 and 74.92, and median translation error 0.07 m versus 0.11 m.
- On NYUv2 zero-shot metric depth, Plana3R achieves Rel 0.132 and delta1 86.4 versus MASt3R's 0.152 and 83.0.
- Single-view plane segmentation on Replica reaches RI 0.89 and VOI 1.62 versus PlaneRecTR's 0.85 and 1.81.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PLANA3R instead of MASt3R | Two-view planar reconstruction · ScanNetV2 | Chamfer: 0.21 -> 0.11 | Helped |
| Use PLANA3R instead of MASt3R | Two-view planar reconstruction · Matterport3D | F-score: 30.01 -> 56.63 | Helped |
| Use PLANA3R instead of MASt3R | Metric monocular depth estimation · NYUv2 | RMSE: 0.51 -> 0.463 | Helped |
| Use PLANA3R instead of PlaneRecTR | Single-view planar reconstruction and segmentation · Replica | RI: 0.85 -> 0.89 | Helped |
| Use PLANA3R instead of MASt3R | Pairwise multi-view pose estimation · ScanNetV2 8-view | RTA@5: 0.2657 -> 0.3935 | Helped |
| Use PLANA3R instead of MASt3R | Pairwise multi-view pose estimation · ScanNetV2 8-view | RRA@5: 0.9828 -> 0.9000 | Hurt |
| Set gradient threshold from 0.5 to 0 (ablation) | PLANA3R · ScanNetV2 reconstruction | F-score: 92.52 -> 93.10 | Helped |
| Set gradient threshold from 0.5 to 10 (ablation) | PLANA3R · ScanNetV2 reconstruction | F-score: 92.52 -> 92.32 | Hurt |
| Set gradient threshold from 0.5 to 0 (ablation) | PLANA3R · NYUv2 depth estimation | RMSE: 0.46 -> 0.45 | Helped |
| Set gradient threshold from 0.5 to 10 (ablation) | PLANA3R · NYUv2 depth estimation | RMSE: 0.46 -> 0.49 | Hurt |
| Remove 0.57M non-overlapping training pairs (ablation) | PLANA3R · ScanNetV2, all overlaps | Mean translation error: 0.13 -> 0.20 | Hurt |
| Remove 0.57M non-overlapping training pairs (ablation) | PLANA3R · MP3D, all overlaps | Mean translation error: 0.45 -> 0.64 | Hurt |
Surprising findings
- Including 0.57M non-overlapping image pairs in training improves performance, enabling pose and geometry inference with little or no view overlap.
- Using only about half the high-resolution primitives yields performance comparable to the full set, while low-resolution-only primitives cause a significant accuracy drop.
Limitations
- No quantitative plane segmentation benchmark with reliable out-of-domain plane annotations; evaluation is largely qualitative.
- Multi-view reconstruction is done pairwise without a single forward pass for three or more views.
- Performance degrades as the overlap ratio between test pairs decreases.
Experimental setup
Datasets: ScanNetV2 ScanNet++ ARKitScenes Habitat Matterport3D NYUv2 Replica 7Scenes
Benchmarks: ScanNetV2 Matterport3D NYUv2 Replica 7-Scenes
Metrics: Chamfer Distance F-score translation error rotation error Rel RMSE delta1 RI VOI SC
Baselines: SparsePlanes PlaneFormers NOPE-SAC MASt3R PlaneNet PlaneAE PlaneRCNN PlaneTR PlaneRecTR
Builds on: PlanarSplatting DUSt3R MASt3R Metric3Dv2
Closest papers by meaning
- PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes (CVPR 2025 · similarity 0.90)
- PlanarGS: High-Fidelity Indoor 3D Gaussian Splatting Guided by Vision-Language Planar Priors (NeurIPS 2025 · similarity 0.89)
- Towards In-the-wild 3D Plane Reconstruction from a Single Image (CVPR 2025 · similarity 0.88)
- AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos (NeurIPS 2024 · similarity 0.86)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.86)
- Rooms from Motion: Un-posed Indoor 3D Object Detection as Localization and Mapping (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.