Research map / 3D vision / Depth and stereo
Bilateral Propagation Network for Depth Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BP-Net, a three-stage multi-scale depth completion network whose first stage propagates depth from nearby sparse measurements using MLP-generated coefficients conditioned on image content and spatial distance.
Problem
Depth completion from sparse LiDAR/SfM depth with a color image; existing SOTA propagation methods rely on initial dense depth from convolutions applied directly to sparse depth, which suffers from valid/invalid ambiguity and poor handling of irregular sparsity.
Approach
Propagate depth at the earliest stage via a bilateral-style learned non-linear model: each target pixel's depth is an softmax-weighted combination of affine transforms of N nearest valid depth points, with coefficients produced by an MLP conditioned on image features and spatial offsets, followed by U-Net multimodal fusion and CSPN++-style refinement in a multi-scale framework.
Key results
- Ranks 1st on KITTI depth completion leaderboard with RMSE 684.90 mm at time of submission.
- Achieves best NYUv2 RMSE of 0.089 m and best delta1.25 of 99.6%.
- Ablation: adding Pre. stage to MF. reduces NYUv2 RMSE from 91.68 to 89.56 mm; full 3-stage model reaches 88.69 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use bilateral preprocessing before multi-modal fusion (ablation) | BP-Net · NYUv2 | RMSE (mm): 91.68 -> 89.56 | Helped |
| Add post-processing to multi-modal fusion (ablation) | BP-Net · NYUv2 | RMSE (mm): 91.68 -> 90.04 | Helped |
| Add bilateral preprocessing rather than post-processing to MF. (ablation) | BP-Net · NYUv2 | RMSE (mm): 90.04 -> 89.56 | Helped |
| Combine preprocessing, multi-modal fusion, and post-processing (ablation) | BP-Net · NYUv2 | RMSE (mm): 91.68 -> 88.69 | Helped |
| Condition propagation coefficients on image content and spatial distance (ablation) | BP-Net bilateral propagation · NYUv2 | RMSE | Helped |
| Add content-conditioned propagation (ablation) | BP-Net bilateral propagation · NYUv2 | RMSE | Helped |
| Add spatially conditioned propagation (ablation) | BP-Net bilateral propagation · NYUv2 | RMSE | Helped |
| Add multi-scale architecture while retaining multi-scale loss (ablation) | BP-Net · NYUv2 | RMSE (mm): 92.46 -> 88.69 | Helped |
| Add multi-scale loss to multi-scale depth estimation (ablation) | BP-Net · NYUv2 | RMSE (mm): 92.46 -> 88.69 | Helped |
| Use multi-scale architecture without multi-scale loss (ablation) | BP-Net · NYUv2 | RMSE: 90.40 -> 92.46 | Hurt |
| Increase the number of neighboring depth points from 1 to 2 (ablation) | BP-Net bilateral propagation · NYUv2 | RMSE | Helped |
| Vary the number of neighboring depth points beyond 2 (ablation) | BP-Net bilateral propagation · NYUv2 | RMSE | No clear effect |
Surprising findings
- High-dimensional position encoding gave little improvement over the simple spatial offset encoding despite added cost.
- Multi-scale architecture without multi-scale loss decreased performance on NYUv2.
Limitations
- Mostly local propagation and convolution; may struggle with long-range information delivery.
- Gains on NYUv2 are smaller than on KITTI due to sparser, more diverse inputs.
Experimental setup
Datasets: NYUv2 KITTI depth completion
Benchmarks: KITTI depth completion benchmark NYUv2
Metrics: RMSE MAE iRMSE iMAE REL delta 1.25
Baselines: S2D CSPN DeepLiDAR CSPN++ GuideNet FCFR ACMNet NLSPN RigNet DySPN BEV@DC CFormer
Builds on: bilateral filtering CSPN++ CSPN U-Net
Closest papers by meaning
- Dynamic Spatial Propagation Network for Depth Completion (AAAI 2022 · similarity 0.90)
- CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion (AAAI 2020 · similarity 0.90)
- Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinement (CVPR 2024 · similarity 0.87)
- DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions (CVPR 2024 · similarity 0.86)
- Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion (CVPR 2025 · similarity 0.86)
- Efficient Indoor Depth Completion Network Using Mask-adaptive Gated Convolution (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.