Research map / 3D vision / Depth and stereo
Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pow3R extends DUSt3R into a versatile 3D regression model that accepts any subset of camera intrinsics, relative pose, and sparse or dense depth alongside RGB images, using random modality dropout during training.
Problem
Feed-forward 3D reconstruction models like DUSt3R only accept RGB images and cannot exploit camera or scene priors (intrinsics, poses, depth) that are often available at test time.
Approach
Inject any subset of auxiliary modalities (intrinsics as ray patches, sparse/dense depth as patchified maps, relative pose as a CLS-token embedding) via lightweight MLP injection into a DUSt3R-style ViT, trained with random modality subsets; also predicts the second image's pointmap in its own frame so relative pose can be obtained via Procrustes alignment in one pass.
Key results
- On Habitat at 224px, Pow3R with intrinsics and depth reaches 98.0 focal acc@1.01 versus 36.6 for DUSt3R when intrinsics are given (plus all modalities: 99.3).
- On DTU MVS, Pow3R with intrinsics and relative pose reduces overall error to 1.115 mm from 1.894 for the reproduced DUSt3R (about 36% relative improvement).
- On Co3Dv2 multi-view pose, Pow3R with known intrinsics reaches mAA(30) 72.5 vs 61.7 for DUSt3R (PnP), with Procrustes alignment roughly 10x faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Provide both camera intrinsics and relative pose | Pow3R · DTU | Overall error (mm): 1.743 -> 1.115 | Helped |
| Provide camera intrinsics | Pow3R · DTU | Overall error (mm): 1.743 -> 1.420 | Helped |
| Provide relative camera pose without intrinsics | Pow3R · DTU | Overall error (mm): 1.743 -> 1.817 | Hurt |
| Use camera intrinsics and relative pose for multiview depth | Pow3R · Average | rel: 3.7 -> 3.2 | Helped |
| Add the X2,2 pointmap prediction head (ablation) | Pow3R · Habitat | Avg.: 77.7 -> 78.3 | Helped |
| Use the X2,2 prediction for single-forward evaluation (ablation) | Pow3R · Habitat | Avg.: 77.6 -> 77.7 | No clear effect |
| Inject auxiliary embeddings in one transformer block instead of embedding before blocks (ablation) | Pow3R · Habitat | Avg.: 77.4 -> 77.7 | No clear effect |
| Inject auxiliary embeddings in four transformer blocks instead of one (ablation) | Pow3R · Habitat | Avg.: 77.7 -> 78.3 | Helped |
Surprising findings
- Adding ground-truth relative pose alone slightly degrades DTU MVS results (-4%), while combined with intrinsics it yields a large gain.
- Naively feeding high-resolution images directly to Pow3R fails dramatically, whereas intrinsics-guided sliding-window inference works well.
Limitations
- Requires known intrinsics or depth at test time for the largest gains; without priors it only matches DUSt3R.
- Procrustes alignment is sensitive to noise and outliers, performing comparably to RANSAC-PnP only in this setting.
- Multi-view global alignment and sliding-window stitching rely on heuristic scale-median blending.
Experimental setup
Datasets: Habitat MegaDepth ARKitScenes Static Scenes 3D BlendedMVS ScanNet++ Co3D-v2 Waymo
Benchmarks: Habitat NYUv2 KITTI ScanNet ETH3D DTU Tanks and Temples Co3Dv2 RealEstate10K
Metrics: absolute relative error inlier ratio focal accuracy RRA RTA mAA accuracy completeness RMSE
Baselines: DUSt3R MASt3R CompletionFormer RobustMVD PoseDiff RelPose++ RayDiff COLMAP
Builds on: DUSt3R CroCo-v2 RayDiffusion MASt3R
Closest papers by meaning
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.89)
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.89)
- MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.89)
- Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation (NeurIPS 2025 · similarity 0.88)
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.88)
- Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.