Research map / 3D vision / Depth and stereo
Multi-view Reconstruction via SfM-guided Monocular Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Murre reconstructs multi-view scenes by feeding SfM sparse depth (densified via KNN plus a distance map) into a fine-tuned Stable Diffusion v2 depth diffusion model, then fusing the metric depth maps with TSDF.
Problem
Learning-based multi-view stereo needs heavy cross-view matching, consuming much GPU memory and failing in sparse-view and low-texture scenes; monocular depth lacks multi-view consistency and metric scale.
Approach
Use an SfM point cloud as an explicit intermediate representation: project it into each view as a sparse depth map, densify with KNN plus a distance map, and use it as a condition for a latent diffusion monocular depth estimator so depth is predicted per view at consistent metric scale without multi-view matching.
Key results
- On DTU (3-view), Murre achieves an average Chamfer distance of 1.42 mm, the best among compared methods (next best IGEV-MVS 2.38 mm, COLMAP 2.56 mm).
- On Replica, F-score reaches 0.853 with 10 denoising steps and 5 ensemble runs, and 0.828 with a single step and single ensemble, versus 0.645 for COLMAP-based SfM input.
- Ablation on KNN densification shows F-score rises from 0.543 without densification to 0.853 with k=3 and distance map.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Murre instead of MonoSDF | Multi-view reconstruction · DTU | Chamfer distance (mm): 1.86 -> 1.42 | Helped |
| Reduce ensemble size from 5 to 1 (ablation) | Murre · Replica | F-score: 0.853 -> 0.850 | No clear effect |
| Reduce denoising steps from 10 to 1 using the LCM-distilled model (ablation) | Murre · Replica | F-score: 0.853 -> 0.829 | Hurt |
| Replace RANSAC alignment with least-squares alignment (ablation) | Murre · Replica | F-score: 0.828 -> 0.812 | Hurt |
| Remove depth alignment (ablation) | Murre · Replica | F-score: 0.828 -> 0.780 | Hurt |
| Use DF-SfM (LoFTR) instead of COLMAP (ablation) | Murre · Replica | F-score: 0.645 -> 0.853 | Helped |
| Replace LoFTR with DKM in DF-SfM (ablation) | Murre · Replica | F-score: 0.853 -> 0.842 | No clear effect |
| Add KNN densification to the sparse SfM depth condition (ablation) | Murre · Replica | F-score: 0.528 -> 0.840 | Helped |
| Add the distance map as a condition (ablation) | Murre · Replica | F-score: 0.783 -> 0.840; 0.753 -> 0.853 | Helped |
| Increase KNN neighborhood size from k=1 to k=3 (ablation) | Murre · Replica | F-score: 0.840 -> 0.853 | Helped |
Surprising findings
- Alignment with SfM depth via RANSAC is nearly unnecessary: without alignment F-score drops only from 0.828 to 0.780, suggesting predicted depth already matches SfM scale.
- Murre remains robust when inference uses a different SfM matcher (DKM) than the training matcher (LoFTR), with F-score 0.842 versus 0.853.
Limitations
- Requires running SfM, which fails with only two views of minimal overlap.
- Designed for static scenes; struggles with moving elements.
- Inference-time cost of diffusion denoising and ensembling is higher than feedforward methods unless reduced steps are used.
Experimental setup
Datasets: Hypersim 3D Ken Burns DTU ScanNet Replica Waymo Open Dataset UrbanScene3D
Benchmarks: DTU ScanNet Replica Waymo Open Dataset UrbanScene3D
Metrics: Chamfer distance F-score RMSE
Baselines: COLMAP RealityCapture VolSDF MonoSDF Marigold Depth-Anything Depth-Anything-v2 Metric3D MVSNet IGEV-MVS DUSt3R NeuralRecon
Builds on: Stable Diffusion v2 Marigold DF-SfM TSDF fusion
Closest papers by meaning
- Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion (CVPR 2025 · similarity 0.88)
- Differentiable Diffusion for Dense Depth Estimation From Multi-View Images (CVPR 2021 · similarity 0.88)
- MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · similarity 0.88)
- Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates (NeurIPS 2025 · similarity 0.88)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.87)
- MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.