Research map / Architectures and efficiency / RNNs and normalization
Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a sparsity-adaptive depth refinement (SDR) framework and masked spatial propagation network (MSPN) that refines monocular depth using a variable number of sparse depth points, also achieving SOTA on standard depth completion.
Problem
Depth completion models are trained with a fixed number of sparse depth measurements and degrade severely when sparsity (number of points or LiDAR lines) changes at test time.
Approach
Refines a monocular depth estimate with a propagation mask initialized from sparse depth locations and updated during iterative refinement, so propagation spreads outward from reliable measured pixels; uses masked pixel-to-window attention as a dynamic filter, with the number of iterations set from sparsity.
Key results
- On NYUv2 ordinary depth completion with 500 sparse points, MSPN achieves RMSE 0.089 m, vs. 0.090 for NLSPN, DySPN, and Zhang et al.
- On KITTI validation with 64 LiDAR lines, MSPN achieves RMSE 835.7 mm vs. 848.7 mm for Zhang et al.
- MSPN outperforms all compared methods across all sparse depth counts on NYUv2 with a single model for SDR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MSPN instead of NLSPN for ordinary depth completion | Depth completion on NYUv2 with 500 sparse depths · NYUv2 (500 sparse depths) | RMSE (m): 0.092 -> 0.089 | Helped |
| Use MSPN instead of NLSPN for ordinary depth completion | Depth completion on NYUv2 with 500 sparse depths · NYUv2 (500 sparse depths) | REL: 0.012 -> 0.012 | No clear effect |
| Use MSPN instead of Zhang et al. | Depth completion on KITTI with 16 LiDAR lines · KITTI validation (16 LiDAR lines) | RMSE (mm): 1218.6 -> 1212.7 | Helped |
| Use MSPN instead of Zhang et al. | Depth completion on KITTI with 16 LiDAR lines · KITTI validation (16 LiDAR lines) | MAE (mm): 337.4 -> 341.8 | Hurt |
| Use MSPN instead of Zhang et al. | Depth completion on KITTI with 64 LiDAR lines · KITTI validation (64 LiDAR lines) | RMSE (mm): 848.7 -> 835.7 | Helped |
| Use MSPN instead of Zhang et al. | Depth completion on KITTI with 64 LiDAR lines · KITTI validation (64 LiDAR lines) | MAE (mm): 215.9 -> 218.5 | Hurt |
| Use MSPN instead of Zhang et al. for depth hole filling | Depth completion in masked NYUv2 regions · NYUv2 depth hole filling (masked areas) | RMSE (m): 0.348 -> 0.325 | Helped |
| Add the propagation mask and update process (ablation) | MSPN for sparsity-adaptive depth refinement · NYUv2 SDR, especially with few sparse depths | RMSE (m) | Helped |
Surprising findings
- Ablating the mask update process severely degrades SDR results, especially with few sparse depths.
Limitations
- Relies on an off-the-shelf monocular depth estimator for the initial depth map in the SDR setting.
- Number of refinement iterations is set by a heuristic formula with a hyperparameter kappa.
Experimental setup
Datasets: NYUv2 KITTI Depth Completion SUN RGB-D
Benchmarks: NYUv2 KITTI Depth Completion validation SUN RGB-D
Metrics: RMSE REL MAE iRMSE iMAE delta threshold accuracy
Baselines: NLSPN DySPN Zhang et al. GuideNet Nconv-CNN PackNet-SAN TWISE S2D
Builds on: spatial propagation network NLSPN DySPN PVT NAF block
Closest papers by meaning
- Flexible Depth Completion for Sparse and Varying Point Densities (CVPR 2024 · similarity 0.88)
- Bilateral Propagation Network for Depth Completion (CVPR 2024 · similarity 0.87)
- Distilling Monocular Foundation Model for Fine-grained Depth Completion (CVPR 2025 · similarity 0.86)
- Improving Depth Completion via Depth Feature Upsampling (CVPR 2024 · similarity 0.86)
- Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation (CVPR 2024 · similarity 0.85)
- Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.