Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Stereo Anywhere couples a RAFT-Stereo-style stereo network with monocular depth from Depth Anything v2 via a second normal-based correlation volume and volume augmentations, trained only on SceneFlow, and introduces the MonoTrap illusion dataset.
Problem
Deep stereo matching generalizes poorly and fails on textureless regions, occlusions, and non-Lambertian surfaces (mirrors, transparencies), while monocular depth foundation models are fooled by perspective illusions.
Approach
A dual-branch stereo network that builds a second correlation volume from normals of monocular VFM depth maps, uses it to scale the monocular depth to metric disparity, and fuses it with the stereo volume via volume augmentations and a mirror-aware truncation, so the model can choose between stereo and mono cues.
Key results
- Zero-shot on Middlebury 2014 bad-2 All of 6.96% vs 9.46% for DLNR (second best).
- On Booster (Q) bad-2 error 9.01% vs 17.84% for RAFT-Stereo baseline; ranks first on Booster online benchmark after fine-tuning.
- On MonoTrap, AbsRel 3.50 vs 5.01 for RAFT-Stereo and 53.46 for Depth Anything v2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add monocular context without retraining the context backbone (ablation) | RAFT-Stereo · Booster (Q) and Middlebury 2014 (H) | bad > 2 All (%): 17.84 -> 15.85; 11.15 -> 14.96 | Mixed |
| Add monocular context and retrain the context backbone (ablation) | RAFT-Stereo · Booster (Q) and Middlebury 2014 (H) | bad > 2 All (%): 17.84 -> 14.94; 11.15 -> 9.62 | Helped |
| Add normals correlation volume and differentiable depth scaling (ablation) | Monocular-context RAFT-Stereo · Booster (Q) | bad > 2 All (%): 14.94 -> 11.33 | Helped |
| Add volume augmentations and truncation (ablation) | Stereo Anywhere · Booster (Q) | bad > 2 All (%): 11.33 -> 9.01 | Helped |
| Add volume augmentations and truncation (ablation) | Stereo Anywhere · Middlebury 2014 (H) | bad > 2 All (%): 7.67 -> 6.96 | Helped |
| Use Stereo Anywhere instead of RAFT-Stereo | RAFT-Stereo · MonoTrap | AbsRel: 5.01 -> 3.50 | Helped |
| Use Stereo Anywhere instead of RAFT-Stereo | RAFT-Stereo · Booster (Q) | bad > 2 (%): 17.84 -> 9.01 | Helped |
Surprising findings
- Off-the-shelf monocular VFMs such as Depth Anything v2 and DepthPro fail badly on MonoTrap illusions, while stereo models are more robust there.
Limitations
- MonoTrap contains only 26 scenes, limiting evaluation breadth.
- Relies on the quality of the monocular VFM; the paper notes monocular priors can mislead stereo in illusions, though the model partially ignores them.
Experimental setup
Datasets: SceneFlow HyperSim Booster training set
Benchmarks: Middlebury 2014 Middlebury 2021 ETH3D KITTI 2012 KITTI 2015 Booster LayeredFlow MonoTrap
Metrics: bad-tau error average pixel error AbsRel RMSE delta<1.05 accuracy
Baselines: RAFT-Stereo PSMNet GMStereo ELFNet PCVNet DLNR Selective-RAFT Selective-IGEV IGEV-Stereo NMRF Depth Anything v2 DepthPro
Builds on: RAFT-Stereo Depth Anything v2 CoEx IGEV-Stereo
Closest papers by meaning
- FoundationStereo: Zero-Shot Stereo Matching (CVPR 2025 · similarity 0.89)
- Zero-Shot Event-Intensity Asymmetric Stereo via Visual Prompting from Image Domain (NeurIPS 2024 · similarity 0.89)
- Any-Stereo: Arbitrary Scale Disparity Estimation for Iterative Stereo Matching (AAAI 2024 · similarity 0.88)
- Stereo Risk: A Continuous Modeling Approach to Stereo Matching (ICML 2024 · similarity 0.88)
- MonSter: Marry Monodepth to Stereo Unleashes Power (CVPR 2025 · similarity 0.88)
- DEFOM-Stereo: Depth Foundation Model Based Stereo Matching (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.