Research map / 3D vision / Depth and stereo
Cross-spectral Gated-RGB Stereo Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fuses high-resolution stereo RCCB (visible/NIR) cameras with stereo gated NIR imaging via a cross-spectral matching module to produce high-resolution long-range stereo depth.
Problem
Gated depth estimation is limited to gated-imager resolution and lacks color information, while RGB stereo fails in low light and texture-less regions; accurate long-range depth at high resolution across all illumination is needed for automotive perception.
Approach
A cross-spectral matching layer that warps RCCB features into the gated frame using calibrated pose plus a learned residual pose refinement (PoseNet with time offset), then fuses aligned features via attention; the stereo network (CREStereo-based) alternates modalities during training with self-supervised and LiDAR losses.
Key results
- Gated RCCB Stereo reaches 2.03 m MAE at night and 1.95 m MAE by day on sparse LiDAR, versus 2.25 m for Gated Stereo in both conditions.
- On accumulated LiDAR for 100-220 m at night, MAE is 11.15 m versus 18.36 m for Gated Stereo, a 39.6% improvement.
- Averaged over day and night, the method reduces MAE by 16.1% relative to Gated Stereo over 0-220 m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine stereo RCCB and gated imaging | Gated RCCB Stereo · Night, accumulated LiDAR, 100 - 220 m | MAE: 18.36 -> 11.15 | Helped |
| combine stereo RCCB and gated imaging | Gated RCCB Stereo · Day, accumulated LiDAR, 100 - 220 m | MAE: 16.79 -> 10.07 | Helped |
| retain full RCCB image resolution (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 2.04 -> 2.03 | No clear effect |
| add cross-spectral training (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 2.18 -> 2.03 | Helped |
| add pose refinement (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 2.23 -> 2.03 | Helped |
| add attention-based feature fusion (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 2.42 -> 2.03 | Helped |
| use MPViT instead of the backbone from [48] (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 2.74 -> 2.03 | Helped |
| use stereo rather than the RCCB-Gated setup (ablation) | Gated RCCB Stereo · Night, LiDAR ground-truth points | MAE: 7.89 -> 2.03 | Helped |
| use stereo rather than monocular gated depth (ablation) | Gated RCCB Stereo · Day, LiDAR ground-truth points | MAE: 6.05 -> 1.95 | Helped |
Surprising findings
- Removing the dual-camera setup (monocular baseline) more than doubles MAE, showing the value of stereo cues.
Limitations
- Evaluation of long-range accuracy relies on accumulated LiDAR ground truth available for only 655 frames.
- Gated Stereo baseline is designed for distances up to 160 m only, so comparison at long range is partly out of its design range.
Experimental setup
Datasets: Walz et al. gated stereo dataset (extended with RCCB stereo)
Benchmarks: Walz et al. gated stereo dataset with LiDAR ground truth Accumulated LiDAR ground truth 0-220 m
Metrics: RMSE MAE ARD delta1 delta2 delta3
Baselines: Gated2Depth Gated2Gated PackNet Monodepth2 SimIPU AdaBins DPT Depthformer PSMNet STTR HSMNet ACVNet
Builds on: CREStereo Gated Stereo MPViT Gated2Gated
Closest papers by meaning
- Gated Fields: Learning Scene Reconstruction from Gated Videos (CVPR 2024 · similarity 0.86)
- Learned Binocular-Encoding Optics for RGBD Imaging Using Joint Stereo and Focus Cues (CVPR 2025 · similarity 0.85)
- Flexible Depth Completion for Sparse and Varying Point Densities (CVPR 2024 · similarity 0.84)
- Snapshot Lidar: Fourier Embedding of Amplitude and Phase for Single-Image Depth Reconstruction (CVPR 2024 · similarity 0.84)
- HELVIPAD: A Real-World Dataset for Omnidirectional Stereo Depth Estimation (CVPR 2025 · similarity 0.83)
- Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.