Research map / 3D vision / Depth and stereo
MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MVSFormer++, which adds cross-view side-view attention to frozen DINOv2 features and uses a vanilla-attention cost volume transformer with normalized 3D positional encoding and adaptive attention scaling for multi-view stereo depth estimation.
Problem
Transformer-based learning MVS methods have not thoroughly investigated how attention should be tailored to different MVS modules (feature encoder vs cost volume regularization), how to inject cross-view information into pretrained ViTs, and how to generalize transformers across varying test resolut
Approach
Use different attention types per module (linear attention with cross-view side-tuning for the DINOv2 feature encoder, vanilla attention for a global cost volume transformer), plus normalized 2D/3D positional encodings, adaptive attention scaling to counter attention dilution at high resolution, and Pre-LN/ALS placement choices.
Key results
- Achieves overall DTU point cloud error of 0.2805 mm, the best among compared methods (MVSFormer 0.289).
- Tanks-and-Temples mean F-scores of 67.03 (Intermediate) and 41.70 (Advanced), state-of-the-art at submission time.
- ETH3D F1-score 82.99, marginally above MVSFormer's 82.85.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace stage-1 3DCNN with CVT, without FPE or AAS (ablation) | MVSFormer++ on DTU · DTU test | Overall (mm): 0.2875 -> 0.2855 | Helped |
| Add FPE to CVT (ablation) | MVSFormer++ on DTU · DTU test | e2: 17.95 -> 13.89 | Helped |
| Add AAS to CVT + FPE (ablation) | MVSFormer++ on DTU · DTU test | e2: 13.89 -> 13.76 | Helped |
| Add SVA to CVT + FPE + AAS (ablation) | MVSFormer++ on DTU · DTU test | e2: 13.76 -> 12.41 | Helped |
| Add Norm&ALS to CVT + FPE + AAS + SVA (ablation) | MVSFormer++ on DTU · DTU test | Overall (mm): 0.2815 -> 0.2805 | Helped |
| Use vanilla attention + AAS instead of linear attention in the cost volume (ablation) | CVT on DTU · DTU test | Overall (mm): 0.2980 -> 0.2847 | Helped |
| Use linear attention instead of vanilla attention in the feature encoder (ablation) | Feature encoder on DTU · DTU test | Overall (mm): 0.2835 -> 0.2805 | Helped |
| Replace feature-encoder Pre-LN with Post-LN (ablation) | MVSFormer++ on DTU · DTU test | Overall (mm): 0.2805 -> 0.2850 | Hurt |
| Replace cost-volume Post-LN with Pre-LN (ablation) | MVSFormer++ on DTU · DTU test | Overall (mm): 0.2805 -> 0.2827 | Hurt |
| Replace FFN with GLU (ablation) | Transformer MLPs on DTU · DTU test | Overall (mm): 0.2805 -> 0.2809 | Hurt |
| Replace standard 2D-PE with normalized 2D-PE (ablation) | TransMVSNet FMT · TransMVSNet DTU test | e2: 23.92 -> 23.15 | Helped |
| Fine-tune all DINOv2-base parameters rather than freeze the backbone (ablation) | MVSFormer on DTU · DTU test | Overall (mm): 0.2875 -> 0.2905 | Hurt |
Surprising findings
- Linear attention performs very poorly in the cost volume regularization but is best for the feature encoder.
- Frozen DINOv2 outperforms full fine-tuning of DINOv2 for MVS, which the authors attribute to degraded generalization from limited DTU data.
Limitations
- Coarse-stage errors on tiny foregrounds accumulate across coarse-to-fine stages.
- Total parameter count rises to 126.95M (39.48M trainable) versus 28.01M for MVSFormer.
Experimental setup
Datasets: DTU BlendedMVS
Benchmarks: DTU Tanks-and-Temples ETH3D
Metrics: accuracy completeness overall error depth error ratio e2/e4/e8 F-score precision recall F1-score
Baselines: CasMVSNet TransMVSNet WT-MVSNet CostFormer MVSFormer UniMVSNet GeoMVSNet RA-MVSNet COLMAP Gipuma
Builds on: MVSFormer DINOv2 MVSNet CasMVSNet FlashAttention
Closest papers by meaning
- MVS2D: Efficient Multi-View Stereo via Attention-Driven 2D Convolutions (CVPR 2022 · similarity 0.91)
- A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding (NeurIPS 2024 · similarity 0.89)
- MVSMamba: Multi-View Stereo with State Space Model (NeurIPS 2025 · similarity 0.89)
- MVSAnywhere: Zero-Shot Multi-View Stereo (CVPR 2025 · similarity 0.89)
- RRT-MVS: Recurrent Regularization Transformer for Multi-View Stereo (AAAI 2025 · similarity 0.89)
- MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.