Research map / 3D vision / Depth and stereo
MonSter: Marry Monodepth to Stereo Unleashes Power
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MonSter, a dual-branch stereo matching network that couples a frozen Depth Anything V2 monocular branch with an IGEV-style stereo branch, iteratively refining each through confidence-guided scale-shift alignment and mono-guided disparity refinement.
Problem
Stereo matching fails in ill-posed regions with limited matching cues (occlusions, textureless areas, reflective surfaces, thin structures, distant objects), and monocular depth priors suffer from scale and shift ambiguities when fused naively.
Approach
Decompose stereo into monocular depth plus per-pixel scale-shift recovery: stereo cues confidence-selected via flow residuals update a per-pixel shift of the monocular disparity (SGA), and the refined monocular disparity guides stereo refinement (MGR), iterated over a few rounds.
Key results
- Achieves 0.37 EPE on Scene Flow, a 21.28% improvement over IGEV and 15.91% over Selective-IGEV, the previous best.
- On ETH3D, Bad 1.0 (Noc) of 0.46 vs 0.91 for LoS (49.45% improvement); ranks 1st across Scene Flow, KITTI 2012, KITTI 2015, Middlebury, and ETH3D leaderboards.
- Zero-shot trained on Scene Flow only, MonSter achieves best generalization across KITTI, Middlebury, and ETH3D; with 4 iterations it still gets 0.42 EPE on Scene Flow (vs 0.47 for IGEV with 32 iterations).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MonSter's monocular-stereo mutual refinement instead of IGEV | IGEV stereo matching · Scene Flow test | EPE (px): 0.47 -> 0.37 | Helped |
| use MonSter instead of IGEV | stereo matching · ETH3D | Bad 1.0 (Noc): 1.12 -> 0.46 | Helped |
| use MonSter instead of IGEV | stereo matching · Middlebury | RMSE: 12.80 -> 6.71 | Helped |
| use MonSter instead of IGEV | stereo matching · KITTI 2015 | D1-all: 1.49 -> 1.33 | Helped |
| use MonSter instead of IGEV | stereo matching · KITTI 2012 | Out-3 (All): 1.44 -> 1.09 | Helped |
| add MGR-based monocular-stereo disparity fusion instead of convolution-based fusion (ablation) | MonSter disparity fusion · Scene Flow test | EPE (px): 0.46 -> 0.43 | Helped |
| add convolution-based scale and shift refinement to Mono+MGR (ablation) | MonSter · Scene Flow test | EPE (px): 0.43 -> 0.42 | Helped |
| add Stereo Guided Alignment (SGA) instead of convolution-based scale and shift refinement (ablation) | Mono+MGR · Scene Flow test | EPE (px): 0.43 -> 0.39 | Helped |
| add feature sharing (ablation) | Mono+MGR+SGA · Scene Flow test | EPE (px): 0.39 -> 0.37 | Helped |
| reduce the iteration count from 32 to 4 (ablation) | MonSter · Scene Flow test | EPE (px): 0.37 -> 0.42 | Hurt |
| replace DepthAnythingV2 with DepthAnythingV1 (ablation) | MonSter monocular depth branch · Scene Flow test | EPE (px): 0.37 -> 0.39 | Hurt |
| replace DepthAnythingV2 with MiDaS (ablation) | MonSter monocular depth branch · Scene Flow test | EPE (px): 0.37 -> 0.41 | Hurt |
Surprising findings
- Parameter count does not determine effectiveness: CroCo-Stereo (437.4M) is larger than MonSter (356.1M) yet performs worse.
- Frozen monocular encoder with shared features improves stereo accuracy; the method works with several monocular models (Depth Anything V1, MiDaS) and still beats IGEV.
Limitations
- Inference time and memory increase substantially (0.37s to 0.64s, 356.1M parameters) due to the ViT-L monocular branch.
- Authors note memory reduction via encoder quantization or distillation is left to future work.
- Reported gains are measured against IGEV-based baselines and depend on a large pretrained monocular model.
Experimental setup
Datasets: Scene Flow CREStereo Tartan Air Sintel Stereo FallingThings InStereo2k ETH3D Middlebury KITTI 2012 KITTI 2015
Benchmarks: Scene Flow KITTI 2012 KITTI 2015 ETH3D Middlebury
Metrics: EPE Bad 1.0 Bad 2.0 RMSE D1-bg D1-all Out-2 Out-3 Out-4 >1px
Baselines: IGEV Selective-IGEV NMRF-Stereo LoS CREStereo CroCo-Stereo ACVNet CFNet RAFT-Stereo GwcNet LEAStereo GANet
Builds on: IGEV Depth Anything V2 RAFT-Stereo DPT
Closest papers by meaning
- UniDepth: Universal Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.88)
- Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail (CVPR 2025 · similarity 0.88)
- Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion (CVPR 2025 · similarity 0.88)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.87)
- Any-Stereo: Arbitrary Scale Disparity Estimation for Iterative Stereo Matching (AAAI 2024 · similarity 0.87)
- Dual-Level Precision Edges Guided Multi-View Stereo with Accurate Planarization (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.