Research map / Video / Video segmentation and tracking
SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SVDC, a lightweight multi-frame video depth completion model for sparse noisy dToF depth fused with RGB, using adaptive frequency selective fusion and a cross-window consistency loss.
Problem
Completing sparse and noisy direct Time-of-Flight (dToF) depth maps in mobile video streams, where multi-frame fusion causes edge blending from misalignment and cross-window predictions flicker.
Approach
An adaptive frequency selective fusion module uses an attention map from a channel-spatial enhancement attention module to pick small kernels for high-frequency (edge) regions and large kernels for smooth regions when fusing multi-frame features, plus a cross-window SILoss that enforces consistency of the same frame predicted in overlapping windows.
Key results
- SVDC reaches RMSE 0.164 m and REL 0.024 on TartanAir, versus 0.183 m and 0.030 for DVDC and 0.352 m for CFormer.
- On Dynamic Replica, SVDC achieves RMSE 0.086 m and OPW 0.171 versus CFormer's 0.127 m and 0.465 OPW.
- CSEA and AFSF together add only about 0.1M parameters over DVDC (22.7M to 22.8M) while lowering TartanAir RMSE from 0.183 to 0.164 m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-frame SVDC instead of per-frame CFormer | Depth completion · TartanAir | RMSE (m): 0.352 -> 0.164 | Helped |
| Use multi-frame SVDC instead of per-frame CFormer | Depth completion · TartanAir | OPW: 0.254 -> 0.159 | Helped |
| Use multi-frame SVDC instead of per-frame CFormer | Depth completion · Dynamic Replica | RMSE (m): 0.127 -> 0.086 | Helped |
| Use multi-frame SVDC instead of per-frame CFormer | Depth completion · Dynamic Replica | OPW: 0.465 -> 0.171 | Helped |
| Add AFSF to DVDC (ablation) | DVDC · TartanAir | RMSE (m): 0.183 -> 0.175 | Helped |
| Add AFSF to DVDC (ablation) | DVDC · TartanAir | REL: 0.030 -> 0.026 | Helped |
| Add AFSF to DVDC (ablation) | DVDC · TartanAir | OPW: 0.166 -> 0.161 | Helped |
| Add CSEA to DVDC+AFSF (ablation) | DVDC+AFSF · TartanAir | RMSE (m): 0.175 -> 0.164 | Helped |
| Add CSEA to DVDC+AFSF (ablation) | DVDC+AFSF · TartanAir | REL: 0.026 -> 0.024 | Helped |
| Add CSEA to DVDC+AFSF (ablation) | DVDC+AFSF · TartanAir | OPW: 0.161 -> 0.159 | Helped |
| Add OPW loss (ablation) | SVDC · Dynamic Replica | OPW (intra-window): 0.145 -> 0.076 | Helped |
| Add OPW loss (ablation) | SVDC · Dynamic Replica | RMSE (m): 0.094 -> 0.096 | Hurt |
Surprising findings
- Cross-window loss slightly reduced intra-window OPW only in combination with the OPW loss; a standalone OPW loss did not improve cross-window consistency.
Limitations
- Relies on a pre-trained optical flow model (SpyNet), which struggles in low light or large motion.
- Sparse and noisy dToF input leads to inefficient information propagation and potential errors.
- Evaluated only on synthetic datasets (TartanAir, Dynamic Replica) with simulated dToF sparse depth.
Experimental setup
Datasets: TartanAir Dynamic Replica MIPI DydToF
Benchmarks: TartanAir Dynamic Replica
Metrics: RMSE REL TEPE OPW
Baselines: BPNet CFormer DVDC
Builds on: DVSR FMNet CBAM SpyNet Selective-Stereo
Closest papers by meaning
- Tri-Perspective View Decomposition for Geometry-Aware Depth Completion (CVPR 2024 · similarity 0.84)
- Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing (NeurIPS 2025 · similarity 0.84)
- Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior (AAAI 2025 · similarity 0.83)
- Robust Depth Enhancement via Polarization Prompt Fusion Tuning (CVPR 2024 · similarity 0.83)
- PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.83)
- Event-Driven Dynamic Scene Depth Completion (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.