Research map / 3D vision / Depth and stereo
IINet: Implicit Intra-inter Information Fusion for Real-Time Stereo Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IINet, a compact 2D implicit network for real-time stereo matching that fuses intra-image and inter-image (score volume) information via FMSV, confidence-based filtering, and residual context-aware upsampling.
Problem
Real-time stereo matching is hard because accurate 3D CNN cost aggregation is computationally expensive; naively replacing 3D networks with 2D implicit networks degrades structure, inter-image information quality, and high-resolution regression.
Approach
Replace explicit 3D cost aggregation with a 2D implicit pipeline: multi-scale score volumes built with a lightweight MLP and top-k seeds, a soft-histogram pretrain loss (KL+BCE) to inject high-res GT into low-res volumes, confidence from average cross-entropy to filter noise, intra-image features fused via U-Net, and residual context-aware upsamplers that refine disparity across scales.
Key results
- On SceneFlow, IINet reaches EPE 0.54 px and 26 ms GPU latency, the best EPE among compared fast methods (FastACV 0.64, HitNet 0.55).
- On the Spring benchmark, IINet achieves 0.76 Abs and 3.78% 1px total, ranking second on the leaderboard and beating fast methods FastACV and CoEx.
- Trained only on SceneFlow, IINet gets 8.5% D1 on KITTI15 and 19.57% 2px on Middlebury, better than FastACV, CoEx, and DPF in generalization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IINet instead of HitNet | IINet · SceneFlow test | EPE (px): 0.55 -> 0.54 | Helped |
| Use IINet instead of CroCo | stereo matching on Spring · Spring benchmark | Abs (px): 0.47 -> 0.76 | Hurt |
| Add intra-image features (ablation) | IINet · SceneFlow test | EPE (px): 0.62 -> 0.54 | Helped |
| Add intra-image features (ablation) | IINet · SceneFlow test | D1 (%): 2.54 -> 2.18 | Helped |
| Add the context-aware upsampler (ablation) | IINet · SceneFlow test | EPE (px): 0.69 -> 0.54 | Helped |
| Add confidence-based filtering alongside the context-aware upsampler (ablation) | IINet · SceneFlow test | EPE (px): 0.73 -> 0.54 | Helped |
| Add the Residual Context-aware Upsampler (ablation) | IINet · SceneFlow test | EPE (px): 0.74 -> 0.54 | Helped |
| Use the full multi-scale score-volume configuration instead of MSV3 (ablation) | IINet · SceneFlow test | EPE (px): 0.74 -> 0.54 | Helped |
| Use the full multi-scale score-volume configuration instead of MSV2 (ablation) | IINet · SceneFlow test | EPE (px): 0.78 -> 0.54 | Helped |
| Use the full multi-scale score-volume configuration instead of MSV1 (ablation) | IINet · SceneFlow test | EPE (px): 0.84 -> 0.54 | Helped |
Surprising findings
- Removing the intra-image information mainly degrades pixel-error metrics (3px and D1) more than EPE, which the authors attribute to intra features helping reconstruct low-confidence regions.
- Reducing the score volumes to a single 1/8 scale only slightly lowers speed (26 to 19 ms) while costing notable accuracy, showing inter information matters but with diminishing speed returns.
Limitations
- Evaluated mainly on SceneFlow with a single fast-method comparison set; Spring results rely on the leaderboard and fine-tuned baselines.
- Does not use Transformer architecture or large-scale pretraining, so it trails CroCo-Stereo on Spring accuracy.
- Relies on a confidence threshold (Pth=0.9) and hand-tuned filtering parameters.
Experimental setup
Datasets: SceneFlow Spring KITTI 2012 KITTI 2015 Middlebury 2014
Benchmarks: SceneFlow Spring KITTI 2012 KITTI 2015 Middlebury 2014
Metrics: EPE D1 1px 3px Abs time (ms)
Baselines: DPF AANet BGNet+ CoEx Fast-ACVNet HitNet ACVNet CroCo-Stereo
Builds on: RAFT NP-CVP-MVSNet SimpleRecon MobileNetV3 U-Net
Closest papers by meaning
- CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (CVPR 2021 · similarity 0.89)
- AANet: Adaptive Aggregation Network for Efficient Stereo Matching (CVPR 2020 · similarity 0.88)
- Iterative Geometry Encoding Volume for Stereo Matching (CVPR 2023 · similarity 0.88)
- Practical Stereo Matching via Cascaded Recurrent Network With Adaptive Correlation (CVPR 2022 · similarity 0.87)
- Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep (AAAI 2024 · similarity 0.87)
- Stereo Risk: A Continuous Modeling Approach to Stereo Matching (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.