Research map / 3D vision / Depth and stereo
Rectification-specific Supervision and Constrained Estimator for Online Stereo Rectification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a matching-based online stereo rectification framework that trains a semi-dense matcher (Efficient LoFTR) with rectification-specific losses derived from camera pose annotations and solves rectifying homographies with a rectification-constrained Levenberg-Marquardt estimator plus RANSAC, and releases a new Perturbed SceneFlow dataset with ground-truth optical flow.
Problem
Online stereo rectification under vibration and temperature-induced calibration drift; existing camera-pose-based methods optimize pose or epipolar constraints rather than rectification constraints, and matchers are not trained for rectification because ground-truth correspondences are unavailable i
Approach
Directly minimize vertical disparity (rectification error) with a non-linear least-squares estimator over rectifying homographies parameterized by Euler angles with known intrinsics, and train the semi-dense matcher using three rectification-specific losses (rectification-constrained classification, rectification error regression, and camera pose auxiliary loss) that need only camera pose annotati
Key results
- Outperforms prior state of the art by 10.7% in vertical flow on Carla-Flowguided and 21.3% on Semi-Truck Highway (0.52 and 0.59 px vs FlowGuidedNet).
- Achieves 79 ms total runtime versus 81 ms for FlowGuidedNet, and lowest among compared methods.
- Downstream stereo depth MAE of 3.20 on Carla-Flowguided vs 4.27 for FlowGuidedNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed rectification-specific framework instead of FAR | Online stereo rectification · Semi-Truck Highway | Vertical Flow: 21.3% | Helped |
| Use the proposed rectification-specific framework instead of FAR | Online stereo rectification · Carla-Flowguided | Vertical Flow: 10.7% | Helped |
| Use the proposed rectification-specific framework instead of EfficientLoFTR + MAGSAC | Stereo depth estimation · Carla-Flowguided | MAE: 5.93 -> 3.20 | Helped |
| Use the proposed rectification-specific framework instead of EfficientLoFTR + MAGSAC | Stereo depth estimation · Carla-Flowguided | AbsRel: 15.20 -> 7.31 | Helped |
| Use a rectification-constrained solver instead of a 5-Point solver (ablation) | Stereo rectification estimator · Semi-Truck Highway | Vertical Flow: 0.72 -> 0.63 | Helped |
| Use a rectification-constrained solver instead of a 5-Point solver (ablation) | Stereo rectification estimator · Carla-Flowguided | Vertical Flow: 0.57 -> 0.55 | Helped |
| Refit the final inliers using the rectification-constrained solver (ablation) | Rectification-constrained estimator · Semi-Truck Highway | Vertical Flow: 0.68 -> 0.63 | Helped |
| Refit the final inliers using the rectification-constrained solver (ablation) | Rectification-constrained estimator · Carla-Flowguided | Vertical Flow: 0.61 -> 0.55 | Helped |
| Replace Lflow with Lerr supervision (ablation) | Semi-dense matcher · Semi-Truck Highway | Vertical Flow: 0.95 -> 0.61 | Helped |
| Replace Lflow with Lerr supervision (ablation) | Semi-dense matcher · Carla-Flowguided | Vertical Flow: 0.58 -> 0.53 | Helped |
| Add Lpose to Lerr supervision (ablation) | Semi-dense matcher · Semi-Truck Highway | Vertical Flow: 0.61 -> 0.60 | Helped |
| Add Lpose to Lerr supervision (ablation) | Semi-dense matcher · Carla-Flowguided | CorrVert: 0.17 -> 0.15 | Helped |
Limitations
- Assumes intrinsics are known and available via offline calibration.
- Requires camera pose annotations for training supervision and a pretrained RAFT model to filter noisy training samples.
Experimental setup
Datasets: Carla-Flowguided Semi-Truck Highway SceneFlow Perturbed SceneFlow
Benchmarks: Carla-Flowguided Semi-Truck Highway Perturbed SceneFlow
Metrics: vertical flow keypoint offset MAE AbsRel runtime camera pose error
Baselines: SIFT + MAGSAC SP+SG + MAGSAC LoFTR + MAGSAC EfficientLoFTR + MAGSAC DirectionNet ViTPose FAR FlowGuidedNet
Builds on: Efficient LoFTR FlowGuidedNet RAFT RANSAC
Closest papers by meaning
- Flow-Guided Online Stereo Rectification for Wide Baseline Stereo (CVPR 2024 · similarity 0.95)
- Any-Stereo: Arbitrary Scale Disparity Estimation for Iterative Stereo Matching (AAAI 2024 · similarity 0.87)
- Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail (CVPR 2025 · similarity 0.86)
- DEFOM-Stereo: Depth Foundation Model Based Stereo Matching (CVPR 2025 · similarity 0.86)
- PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching (NeurIPS 2025 · similarity 0.86)
- RobIA: Robust Instance-aware Continual Test-time Adaptation for Deep Stereo (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.