Research map / 3D vision / Depth and stereo
MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MV-DUSt3R and MV-DUSt3R+, single-stage feed-forward networks that jointly process many unposed views (up to 24) in one pass, removing DUSt3R's global optimization, with Gaussian heads for novel view synthesis.
Problem
Pose-free multi-view scene reconstruction from sparse views: DUSt3R-style methods process pairs and need a costly global optimization that cannot fix pairwise errors, failing on larger scenes with more views.
Approach
Multi-view decoder blocks exchange tokens across all views relative to a reference view, so pointmaps are predicted in one reference frame without global alignment; MV-DUSt3R+ adds multiple reference-view paths fused by cross-reference-view blocks, and Gaussian heads enable novel view synthesis.
Key results
- MV-DUSt3R runs 48-78x faster than DUSt3R for 4-24 views, and MV-DUSt3R+ runs 8-14x faster while reducing Chamfer distance on large scenes with 24 views (e.g., 2.6x lower CD on HM3D vs MV-DUSt3R).
- On HM3D with 24 views, MV-DUSt3R+ achieves 2.0x lower mAE than DUSt3R for pose estimation.
- MV-DUSt3R+ reconstructs 24-view input for average 17.9 m2 HM3D scenes in under 2 seconds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-view decoding instead of pairwise reconstruction with global optimization | MV-DUSt3R · HM3D, 4 views | CD: 5.6(2.3) -> 2.0(1.1) | Helped |
| use multi-view decoding instead of pairwise reconstruction with global optimization | MV-DUSt3R · ScanNet, 4 views | CD: 4.0(0.4) -> 2.0(0.4) | Helped |
| use multi-view decoding instead of pairwise reconstruction with global optimization | MV-DUSt3R · MP3D, 4 views | CD: 40.0(5.3) -> 25.3(4.1) | Helped |
| add cross-reference-view blocks and multiple reference paths | MV-DUSt3R+ · HM3D, 24 views | CD: 10.0(3.5) -> 3.9(2.0) | Helped |
| add cross-reference-view blocks and multiple reference paths | MV-DUSt3R+ · ScanNet, 24 views | CD: 2.7(0.9) -> 1.7(0.7) | Helped |
| add cross-reference-view blocks and multiple reference paths | MV-DUSt3R+ · MP3D, 24 views | CD: 38.6(13.9) -> 22.0(5.9) | Helped |
| add Gaussian heads (ablation) | MV-DUSt3R+ · HM3D, 4 views | ND: 0.9 -> 1.0 | Hurt |
| add Gaussian heads (ablation) | MV-DUSt3R+ · HM3D, 24 views | ND: 2.1 -> 2.1 | No clear effect |
| train 1-stage on 4 views rather than 8 views (ablation) | MV-DUSt3R+ · HM3D, 24 views | ND: 2.1 -> 17.7 | Hurt |
| fine-tune using mixed input-view counts (ablation) | MV-DUSt3R+ · HM3D, 24 views | ND: 2.1 -> 1.7 | Helped |
| add a Gaussian splatting head and jointly train for rendering | MV-DUSt3R · HM3D, 4 views | PSNR: 16.0 -> 19.9 | Helped |
Surprising findings
- Spann3R performs worse than all other methods and often fails to reconstruct scenes from sparse views.
- Adding Gaussian heads does not significantly improve or degrade MVS reconstruction performance.
Limitations
- Generative/image priors not exploited, so unseen regions are hard to reconstruct from few views
- Reference-view selection still matters; oracle selection gives a noticeable gap for MV-DUSt3R
- Trained at low 224x224 resolution
Experimental setup
Datasets: ScanNet ScanNet++ HM3D Gibson MP3D
Benchmarks: HM3D ScanNet MP3D
Metrics: Chamfer Distance Normalized Distance DistanceAccu@0.2 RRE RTE mAE PSNR SSIM LPIPS reconstruction time
Baselines: DUSt3R Spann3R PoseDiffusion MASt3R-SfM
Builds on: DUSt3R MASt3R Gaussian splatting ViT
Closest papers by meaning
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.94)
- MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.93)
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.91)
- Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model (ICLR 2025 · similarity 0.90)
- Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors (CVPR 2025 · similarity 0.89)
- Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.