Research map / Point clouds / Point cloud classification
DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeCoTR, which upgrades the S2D depth completion baseline with 2D self-attention and then uplifts features to a normalized 3D point cloud processed by point transformer layers with global attention, achieving SOTA on NYUv2 without iterative spatial propagation.
Problem
Image-guided depth completion from sparse depth and RGB, where prior methods rely on 2D feature processing or iterative spatial propagation and 3D methods suffer from extremely sparse input points.
Approach
Enhance a convolutional baseline (S2D) with efficient 2D self-attention in bottleneck and skip connections to get accurate initial depths, then un-project the guidance features into a dense 3D point cloud, normalize it to a unit ball, and apply neighborhood vector attention plus downsampled global attention in 3D.
Key results
- On NYU Depth V2, DeCoTR w/ global attention reaches RMSE 0.086, the lowest among compared methods (CompletionFormer and PointDC at 0.090 and 0.089).
- S2D-TR improves S2D's NYUD RMSE from 0.204 to 0.094 with about 50% less computation than CompletionFormer.
- Zero-shot on DDAD from KITTI-trained models, DeCoTR gets RMSE 399.2 vs NLSPN 701.9 and CompletionFormer 889.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add efficient 2D attention to S2D features (ablation) | S2D · NYUD-v2 | RMSE: 0.204 -> 0.094 | Helped |
| add 3D-TR to the S2D baseline (ablation) | S2D · NYUD-v2 | RMSE: 0.204 -> 0.091 | Helped |
| add neighborhood-based 3D cross-attention after S2D-TR (ablation) | S2D-TR · NYUD-v2 | RMSE: 0.094 -> 0.091 | Helped |
| normalize the uplifted point cloud (ablation) | DeCoTR · NYUD-v2 | RMSE: 0.089 -> 0.087 | Helped |
| add efficient global attention on downsampled 3D points (ablation) | DeCoTR · NYUD-v2 | RMSE: 0.087 -> 0.086 | Helped |
| replace positional-embedding multiplier with conventional relative-position embedding | 3D-TR · Depth completion | depth completion accuracy | No clear effect |
| use DeCoTR instead of CompletionFormer | NYUD-v2-trained depth completion models · ScanNet-v2 zero-shot | RMSE: 0.194 -> 0.188 | Helped |
| use DeCoTR instead of NLSPN | KITTI-trained depth completion models · DDAD zero-shot | RMSE: 701.9 -> 399.2 | Helped |
Surprising findings
- Directly applying 3D transformer layers on the original S2D network did not outperform GraphCSPN or CompletionFormer, showing that accurate initial depth is necessary.
- CompletionFormer has slightly lower RMSE on KITTI-DC test but much worse zero-shot generalization on DDAD.
Limitations
- Zero-shot DDAD accuracy is weaker for far-range depths beyond the KITTI training range of 80 m.
- On KITTI-DC test, RMSE is higher than CompletionFormer (717.07 vs 708.87).
Experimental setup
Datasets: NYU Depth V2 KITTI Depth Completion ScanNet-v2 DDAD
Benchmarks: NYU Depth V2 KITTI Depth Completion ScanNet-v2 DDAD
Metrics: RMSE Abs Rel delta < 1.25 MAE iRMSE iMAE
Baselines: S2D DeepLiDAR CSPN DepthNormal ACMNet GuideNet TWISE NLSPN RigNet DySPN CompletionFormer PRNet
Builds on: S2D Point Transformer CompletionFormer NLSPN GraphCSPN
Closest papers by meaning
- Point Cloud Completion by Skip-Attention Network With Hierarchical Folding (CVPR 2020 · similarity 0.90)
- PointAttN: You Only Need Attention for Point Cloud Completion (AAAI 2024 · similarity 0.88)
- PointCFormer: A Relation-Based Progressive Feature Extraction Network for Point Cloud Completion (AAAI 2025 · similarity 0.88)
- Improving Depth Completion via Depth Feature Upsampling (CVPR 2024 · similarity 0.88)
- Tri-Perspective View Decomposition for Geometry-Aware Depth Completion (CVPR 2024 · similarity 0.87)
- DC-PCN: Point Cloud Completion Network with Dual-Codebook Guided Quantization (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.