Research map / 3D vision / Depth and stereo
MVSAnywhere: Zero-Shot Multi-View Stereo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MVSAnywhere, a transformer-based multi-view stereo model that generalizes zero-shot across domains and depth ranges by combining monocular ViT features with an adaptive, view-count- and scale-agnostic cost volume.
Problem
Multi-view stereo depth estimation typically fails to generalize across domains (indoor vs outdoor) and requires known depth ranges and fixed view counts, limiting use as a general-purpose method.
Approach
Combines a cost volume patchifier that tokenizes the multi-view cost volume guided by monocular ViT features, a view-count-agnostic weighted-sum metadata aggregation with scale normalization, and a cascaded cost volume that infers depth range from camera poses at test time.
Key results
- On RMVDB zero-shot depth, MVSA achieves average rel 2.7 and tau 77.0, outperforming the Robust MVD Baseline (6.3 / 56.0) and MAST3R with triangulation (3.4 / 70.1).
- On ScanNet Mesh Evaluation, MVSA reaches F-score 0.652 without training on ScanNet, outperforming many ScanNet-trained methods such as ATLAS, NeuralRecon and 3DVNet.
- On the RMVDB variant, MVSA achieves rel 3.22 / tau 69.45 on ScanNet and 1.27 / 93.24 on ETH3D, beating the baselines on both.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ViT-B instead of ViT Small (ablation) | MVSA · RMVDB average | rel: 3.40 -> 3.00 | Helped |
| use the proposed architecture instead of [59]'s fully-convolutional architecture (ablation) | MVSA · RMVDB average | rel: 3.39 -> 3.00 | Helped |
| add training-time noise to the ground-truth range (ablation) | MVSA · RMVDB average | rel: 5.32 -> 3.00 | Helped |
| use pretrained Depth Anything V2 weights (ablation) | MVSA · RMVDB average | rel: 3.24 -> 3.00 | Helped |
| use adaptive depth bins instead of fixed bins [0-100m] (ablation) | MVSA · RMVDB average | rel: 3.37 -> 3.00 | Helped |
| use the Mono/Multi Cue Combiner ViT instead of CNN layers (ablation) | MVSA · RMVDB average | rel: 3.41 -> 3.00 | Helped |
| add test-time cost-volume bin refinement (ablation) | MVSA · RMVDB average | rel: 4.26 -> 3.00 | Helped |
| use the proposed cost-volume patchifier instead of naive patchification (ablation) | MVSA · RMVDB average | rel: 3.11 -> 3.00 | Helped |
Surprising findings
- MVSA, never trained on ScanNet, outperforms many methods trained specifically on ScanNet for mesh reconstruction.
- Depth Pro produces sharp edges but frequently displays incorrect depth scaling.
Limitations
- No enforcement of temporal consistency across frames.
- Requires known camera intrinsics and poses, like traditional MVS.
- Ablations are run at reduced resolution and without metadata for efficiency.
Experimental setup
Datasets: Hypersim TartanAIR BlendedMVG MatrixCity VKITTI2 Dynamic Replica MVSSynth SAIL-VOS 3D
Benchmarks: Robust Multi-View Depth Benchmark (RMVDB) ScanNet Mesh Evaluation ETH3D variant of RMVDB
Metrics: absolute relative error (rel) inlier percentage tau (1.03) chamfer distance F-score precision recall
Baselines: MAST3R Depth Anything V2 Depth Pro Metric3D UniDepth MVSFormer++ Robust MVD Baseline MVS2D Fast-MVSNet SimpleRecon
Builds on: SimpleRecon Depth Anything V2 DINOv2 MAST3R DeepVideoMVS
Closest papers by meaning
- MVS2D: Efficient Multi-View Stereo via Attention-Driven 2D Convolutions (CVPR 2022 · similarity 0.89)
- MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo (ICLR 2024 · similarity 0.89)
- Generalized Binary Search Network for Highly-Efficient Multi-View Stereo (CVPR 2022 · similarity 0.89)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.87)
- MVSMamba: Multi-View Stereo with State Space Model (NeurIPS 2025 · similarity 0.87)
- A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.