Research map / 3D vision / Depth and stereo

DUSt3R: Geometric 3D Vision Made Easy

CVPR 2024 · 3D vision › Depth and stereo · approach: Feed-forward 3D reconstruction · 545 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DUSt3R, a transformer that regresses pointmaps from an unposed, uncalibrated image pair (and globally aligns many pairs), unifying monocular, binocular and multi-view 3D tasks without camera parameters.

Problem

Multi-view 3D reconstruction in the wild normally requires estimated camera intrinsics and extrinsics, and classical SfM/MVS pipelines are brittle and sequential. The paper asks whether dense 3D geometry can be regressed directly from unconstrained images without camera calibration or pose.

Approach

Regress two pointmaps expressed in the same frame of the first image with a Siamese ViT encoder and cross-attention decoders trained by a confidence-weighted 3D regression loss; then recover matches, intrinsics, poses and depth from pointmaps, and fuse many pairs with a 3D-space global alignment instead of reprojection-error bundle adjustment.

Key results

• On Map-free relocalization test set, DUSt3R reaches 50.3% pose precision and 0.697 VCRE AUC, outperforming RoMa (45.6%, 0.669).
• On CO3Dv2 multi-view pose with global alignment, DUSt3R 512 reaches RRA@15 96.2 and mAA(30) 67.7 on RealEstate10K, beating PoseDiffusion (80.5 RRA@15 on CO3Dv2).
• On DTU multi-view 3D reconstruction without camera poses, DUSt3R 512 achieves overall error 1.741 mm (zero-shot), behind the best methods that use GT poses.

Measured effects reported in the paper

Surprising findings

• Pointmap-based matching from DUSt3R outperforms dedicated matchers such as RoMa and LoFTR in map-free localization reprojection metrics, despite not being trained for matching.
• On DTU, the method is less accurate than specialized methods that use ground-truth poses, with accuracy 2.677 vs ~0.3 mm for leading MVS methods.

Limitations

• Regression-based outputs are less accurate than triangulation-based methods on fine-grained reconstruction (e.g., DTU accuracy).
• Pointmaps are regressed up to an unknown scale and the model does not enforce explicit geometric constraints, so outputs may not match a physical camera model exactly.
• Global alignment is needed to fuse many views; pairwise prediction alone does not give multi-view consistency.

Experimental setup

Datasets: Habitat MegaDepth ARKitScenes Static Scenes 3D BlendedMVS ScanNet++ CO3D-v2 Waymo

Benchmarks: Map-free relocalization CO3Dv2 RealEstate10K KITTI DDAD NYUv2 BONN TUM DTU ETH3D ScanNet Tanks and Temples

Metrics: VCRE pose accuracy AUC relative rotation accuracy relative translation accuracy mAA(30) AbsRel delta 1.25 accuracy accuracy completeness

Baselines: RoMa LoFTR SuperPoint-SuperGlue PoseDiffusion RelPose SlowTv COLMAP Vis-MVSNet

Builds on: CroCo DPT ViT bundle adjustment

Closest papers by meaning

• MUSt3R: Multi-view Network for Stereo 3D Reconstruction (CVPR 2025 · similarity 0.95)
• MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.94)
• Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (CVPR 2025 · similarity 0.93)
• Test3R: Learning to Reconstruct 3D at Test Time (NeurIPS 2025 · similarity 0.90)
• Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors (CVPR 2025 · similarity 0.89)
• MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion (ICLR 2025 · similarity 0.89)