Research map / 3D vision / Depth and stereo
Learning Intra-view and Cross-view Geometric Knowledge for Stereo Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICGNet, which adds intra-view (interest point detection maps) and cross-view (interest point matching assignments) geometric losses to stereo matching networks, used only during training.
Problem
Stereo matching networks use geometric knowledge mainly from single views (edges, normals) and ignore cross-view factors like occlusion and matching uniqueness.
Approach
Distill intra-view knowledge from a pretrained SuperPoint interest point detector and cross-view knowledge from a pretrained SuperGlue matcher into a stereo network via training-only decoders and losses, adding zero inference overhead.
Key results
- On SceneFlow with IGEV-Stereo base, EPE improves from 0.479 to 0.447 px (state-of-the-art among compared methods).
- On cross-domain generalization, ICGNet improves IGEV-Stereo on Middlebury 2014 (H) EPE from 0.91 to 0.82 and on KITTI 2012 EPE from 1.04 to 0.99.
- With GwcNet base on SceneFlow, EPE drops from 0.765 to 0.597 px, a reported over 20% improvement.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add intra-view geometric loss (ablation) | GwcNet · SceneFlow | EPE (px): 0.765 -> 0.615 | Helped |
| add soft cross-view geometric loss (ablation) | GwcNet with intra-view loss · SceneFlow | EPE (px): 0.615 -> 0.608 | Helped |
| add hard cross-view geometric loss (ablation) | GwcNet with intra-view and soft cross-view losses · SceneFlow | EPE (px): 0.608 -> 0.597 | Helped |
| add intra-view geometric loss (ablation) | IGEV-Stereo · SceneFlow | EPE (px): 0.479 -> 0.452 | Helped |
| add soft cross-view geometric loss (ablation) | IGEV-Stereo with intra-view loss · SceneFlow | EPE (px): 0.452 -> 0.449 | Helped |
| add hard cross-view geometric loss (ablation) | IGEV-Stereo with intra-view and soft cross-view losses · SceneFlow | EPE (px): 0.449 -> 0.447 | Helped |
| use 2 rather than 1 intra-view decoder layer (ablation) | GwcNet with ICGNet · SceneFlow | EPE (px): 0.631 -> 0.615 | Helped |
| increase intra-view decoder from 2 to 4 layers (ablation) | GwcNet with ICGNet · SceneFlow | EPE (px): 0.615 -> 0.623 | Hurt |
| use 4 rather than 0 cross-view decoder layers (ablation) | GwcNet with ICGNet · SceneFlow | EPE (px): 0.617 -> 0.597 | Helped |
| increase cross-view decoder from 4 to 6 layers (ablation) | GwcNet with ICGNet · SceneFlow | EPE (px): 0.597 -> 0.606 | Hurt |
| apply ICGNet geometric knowledge learning | IGEV-Stereo · KITTI 2012 cross-domain | EPE (px): 1.04 -> 0.99 | Helped |
| apply ICGNet geometric knowledge learning | IGEV-Stereo · KITTI 2015 cross-domain | EPE (px): 1.21 -> 1.16 | Helped |
Surprising findings
- Intra-view geometric loss improves occluded-region disparity more than non-occluded (ΔOcc 6.33% vs ΔNoc 3.75% in the ablation table).
Limitations
- Relies on pretrained SuperPoint and SuperGlue, so training requires these external models.
- Gains on strong base models such as IGEV-Stereo are modest (e.g., SceneFlow EPE 0.479 to 0.447).
Experimental setup
Datasets: SceneFlow KITTI 2012 KITTI 2015 Middlebury 2014
Benchmarks: SceneFlow KITTI 2012 KITTI 2015 Middlebury 2014
Metrics: EPE >3px error D1-all 2px error
Baselines: EdgeStereo LEAStereo ACVNet CREStereo RAFT-Stereo AcfNet HITNet GANet GwcNet IGEV-Stereo PSMNet CFNet
Builds on: IGEV-Stereo GwcNet SuperPoint SuperGlue
Closest papers by meaning
- All-in-One: Transferring Vision Foundation Models into Stereo Matching (AAAI 2025 · similarity 0.86)
- Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep (AAAI 2024 · similarity 0.85)
- EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding (NeurIPS 2024 · similarity 0.85)
- IINet: Implicit Intra-inter Information Fusion for Real-Time Stereo Matching (AAAI 2024 · similarity 0.85)
- Bridging Viewpoint Gaps: Geometric Reasoning Boosts Semantic Correspondence (CVPR 2025 · similarity 0.85)
- Hierarchical Object-Aware Dual-Level Contrastive Learning for Domain Generalized Stereo Matching (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.