Research map / Label-efficient and robust learning / Recognition and long-tail

Robust Synthetic-to-Real Transfer for Stereo Matching

CVPR 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Correspondence and causal representation learning · 16 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DKT, a fine-tuning framework for synthetic-pretrained stereo networks that uses a frozen Teacher, an EMA Teacher, and a Student with Filter-and-Ensemble refinement of GT and pseudo labels to preserve domain generalization while adapting to real data.

Problem

Fine-tuning stereo matching networks pre-trained on synthetic data with real ground truth degrades their robustness to unseen domains.

Approach

Divide pixels into consistent and inconsistent regions by the difference between GT and pseudo label, and use an EMA Teacher's predictions to dynamically filter and ensemble GT and PL (Filter and Ensemble module) during fine-tuning, so the Student learns target-domain knowledge without overfitting GT details or learning new knowledge without regularization.

Key results

• On KITTI-fine-tuned IGEV-Stereo, DKT-IGEV reduces DrivingStereo average error from 2.70 (GT fine-tuned baseline reference) to 2.22 and keeps Middlebury/ETH3D/Booster unseen-domain errors low (e.g., Booster 15.30 vs 18.33 for IGEV-Stereo).
• Fine-tuning with GT on Booster severely degrades zero-shot generalization (e.g., Middlebury error 93.31 for GT(valid) training on Booster in the ablation), while DKT keeps unseen errors low (Booster-fine-tuned DKT-IGEV Middlebury 14.11 vs 7.06 zero-shot range in table 5 context).
• DKT is reported as the only published method that generalizes well to Middlebury, ETH3D, and Booster after fine-tuning on KITTI or Booster.

Measured effects reported in the paper

Surprising findings

• Fine-tuning with pseudo labels preserves domain generalization better than fine-tuning with ground truth, even though PL is less accurate on the target domain.
• Fine-tuning on only the inconsistent GT region causes catastrophic degradation, even on the target domain.

Limitations

• Target-domain accuracy is slightly worse than the base networks on KITTI.
• Degradation is more severe on challenging scenes such as Booster with transparent and mirror surfaces; PL alone cannot handle these cases well.
• Requires a frozen Teacher and EMA Teacher alongside the Student, adding training overhead.

Experimental setup

Datasets: SceneFlow KITTI 2012 KITTI 2015 DrivingStereo Booster Middlebury ETH3D

Benchmarks: KITTI 2012 KITTI 2015 DrivingStereo Middlebury ETH3D Booster

Metrics: percentage of pixels with error above threshold (D1/bad-pixe

Baselines: IGEV-Stereo RAFT-Stereo PSMNet GWCNet GANet PCWNet CGF-ACV DLNR CFNet PCVNet

Builds on: IGEV-Stereo Mean Teacher Dark Knowledge distillation RAFT-Stereo

Closest papers by meaning

• Revisiting Domain Generalized Stereo Matching Networks From a Feature Consistency Perspective (CVPR 2022 · similarity 0.91)
• Hierarchical Object-Aware Dual-Level Contrastive Learning for Domain Generalized Stereo Matching (NeurIPS 2024 · similarity 0.89)
• Consistency-aware Self-Training for Iterative-based Stereo Matching (CVPR 2025 · similarity 0.87)
• Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep (AAAI 2024 · similarity 0.87)
• DualNet: Robust Self-Supervised Stereo Matching with Pseudo-Label Supervision (AAAI 2025 · similarity 0.86)
• Stereo Risk: A Continuous Modeling Approach to Stereo Matching (ICML 2024 · similarity 0.86)