Research map / 3D vision / Depth and stereo
MUSt3R: Multi-view Network for Stereo 3D Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the pairwise DUSt3R stereo reconstruction model to a symmetric, memory-based multi-view network (MUSt3R) that predicts pointmaps in a common frame for offline SfM and online visual odometry.
Problem
DUSt3R processes image pairs, so the number of pairs grows quadratically and predictions must be globally aligned, which is costly and slow for large image collections and online video.
Approach
Make DUSt3R symmetric with a single shared-weight Siamese decoder and head, allow cross-attention across N views, and add an iteratively updated KV-cache-like memory of multi-layer tokens plus a global 3D feedback injection, so that pointmaps are predicted directly in a global frame, with a dual local/global pointmap head that gives fast Procrustes-based relative pose.
Key results
- On 11 TUM RGB-D VO sequences, MUSt3R reaches 8.4 cm average ATE at about 8.4 FPS, the best average among dense unconstrained methods versus Spann3R (11.1 cm? no: Spann3R average listed 47.4 cm).
- On CO3Dv2 relative pose, MUSt3R-512 reaches 97.0 RRA@15 and 84.1 mAA(30) versus 77.2 mAA(30) for DUSt3R-512 and 70.4 for Spann3R.
- On ETH3D SLAM, MUSt3R averages 10.0 cm ATE at around 10 FPS, about 50% more accurate than Spann3R.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add rendering after causal prediction (ablation) | MUSt3R-C · TUM RGB ATE RMSE (11 sequences) | ATE RMSE [cm]: 6.0 -> 4.5 | Helped |
| add rendering after causal prediction (ablation) | MUSt3R-C · TUM RGB ATE RMSE (11 sequences) | speed: 11.1 -> 8.4 | Hurt |
| replace Spann3R with MUSt3R | uncalibrated visual odometry · TUM RGBD tracking accuracy | ATE RMSE [cm]: 64.8 -> 14.4 | Helped |
| replace Spann3R with MUSt3R | uncalibrated visual odometry · ETH3D SLAM | ATE RMSE [cm]: 16.8 -> 10.0 | Helped |
| use Procrustes alignment instead of PnP for pose estimation (ablation) | MUSt3R-512 · CO3Dv2 | mAA@30: 84.1 -> 78.3 | Hurt |
| use Procrustes alignment instead of PnP for pose estimation (ablation) | MUSt3R-512 · CO3Dv2 | speed (FPS): 4.1 -> 32.9 | Helped |
| use MUSt3R-224 instead of Spann3R | multi-view depth estimation · Multi-view depth average | rel: 5.0 -> 4.7 | Helped |
| use MUSt3R-512 instead of DUSt3R-512 | multi-view depth estimation · Multi-view depth average | τ: 68.2 -> 66.2 | Hurt |
| increase input resolution from 224 to 512 (ablation) | MUSt3R · 7-Scenes | normal consistency (mean): 0.665 -> 0.617 | Hurt |
Surprising findings
- Procrustes alignment for relative pose gives almost the same accuracy as PnP while being nearly an order of magnitude faster (32.9 vs 4.1 FPS on RealEstate10K-style evaluation).
- Spann3R is penalized by very high errors on several long TUM sequences, while the causal MUSt3R-C variant performs well on both short and long sequences.
Limitations
- Performance degrades on sequences where views drift too far from the first view.
- Memory grows linearly with the number of images, requiring heuristic memory selection for large collections.
Experimental setup
Datasets: Habitat ARKitScenes BlendedMVS MegaDepth Static Scenes 3D ScanNet++ CO3D-v2 Map-free WildRGB-D Virtual KITTI Unreal4K DL3DV
Benchmarks: TUM RGBD ETH3D SLAM CO3Dv2 RealEstate10K 7-Scenes NRGBD DTU KITTI ScanNet ETH3D Tanks and Temples
Metrics: ATE RMSE FoV error scale error RRA@15 RTA@15 mAA(30) Acc Comp NC absolute relative error
Baselines: DUSt3R Spann3R MASt3R-SfM DROID-VO DPVO DSO ORB-SLAM3 COMO GlORIE-VO MonoGS DeepFactors DepthCov
Builds on: DUSt3R MASt3R Spann3R CroCo v2
Closest papers by meaning
- DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 · similarity 0.95)
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.93)
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.92)
- Test3R: Learning to Reconstruct 3D at Test Time (NeurIPS 2025 · similarity 0.89)
- Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors (CVPR 2025 · similarity 0.89)
- Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.