Research map / 3D vision / Depth and stereo
iToF-flow-based High Frame Rate Depth Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes iToF-flow, decomposing iToF measurement variation into cross-mode (LLT-based) and uni-mode (photometric correction) flows to enable high-frame-rate, motion-robust depth extraction from alternating 4-phase iToF measurements.
Problem
Depth estimation with indirect time-of-flight (iToF) cameras degrades in dynamic scenes because the four phase measurements are captured at different times, causing motion artifacts and limiting depth frame rate.
Approach
Models iToF measurement variation as cross-mode flow (a motion-insensitive local linear transform between phase modes plus optical flow for pixel shift) and uni-mode flow (depth-dependent photometric residual derived from a first-order Taylor expansion), implemented as LCTM and UPCM modules that propagate absent phase measurements to each time instant.
Key results
- Proposed method reduces depth MAE from 7.58 cm (FFN) to 4.72 cm (about 37-38% improvement) and achieves the lowest four-mode MAE of 1.51.
- Runtime is 0.081 s versus 0.32 s for FFN, roughly a quarter, with MOM at 0.028 s being faster.
- Ablation shows adding LMM, SFE, and UPCM progressively lowers MAEd from 6.24 to 4.72 cm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iToF flow-based depth extraction instead of FFN | iToF depth reconstruction · iToF test set | MAEd (cm): 7.58 -> 4.72 | Helped |
| Use iToF flow-based depth extraction instead of FFN | iToF measurement reconstruction · iToF test set | MAEp: 3.79 -> 1.51 | Helped |
| Use iToF flow-based depth extraction instead of FFN | iToF depth reconstruction · iToF test set | Time (s): 0.32 -> 0.081 | Hurt |
| Add SFE to LMM (ablation) | LMM-based cross-mode transfer · iToF test set | MAEd (cm): 6.24 -> 5.83 | Helped |
| Add UPCM to LMM (ablation) | LMM-based cross-mode transfer · iToF test set | MAEd (cm): 6.24 -> 5.51 | Helped |
| Add UPCM to LMM + SFE (ablation) | iToF flow depth extraction network · iToF test set | MAEd (cm): 5.83 -> 4.72 | Helped |
| Add LMM to SFE + UPCM (ablation) | iToF flow depth extraction network · iToF test set | MAEd (cm): 5.51 -> 4.72 | Helped |
Surprising findings
- The proposed method has more parameters (7.75M) than FFN (1.37M), yet runs faster than FFN and RAFT because of parallelism.
- MOM is faster (0.028 s) than the proposed method but has much higher depth error (14.28 cm).
Limitations
- Relies on the first-order Taylor approximation and small-depth-change assumption (d much larger than delta d) for uni-mode photometric model.
- Local linear transform assumes two-feature property holds locally; invalid in few regions, mitigated by optical flow.
- Evaluation mainly on simulated data with limited real-world examples (two scenes, qualitative only).
Experimental setup
Datasets: CB-ToF CB-ToF-Extension Matterport3D YCB-V
Benchmarks: iToF synthetic dynamic scenes (20MHz and 30MHz) real-world QVGA Sony IMX518 captures at 30MHz
Metrics: MAE of depth (cm) MAE of four-mode measurements mask rate parameters (M) runtime (s)
Baselines: SDE RAFT FFN MOM
Builds on: FFN MOM RAFT EDB-based optical flow
Closest papers by meaning
- CH3Depth: Efficient and Flexible Depth Foundation Model with Flow Matching (CVPR 2025 · similarity 0.83)
- Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing (NeurIPS 2025 · similarity 0.83)
- Explicit Depth-Aware Blurry Video Frame Interpolation Guided by Differential Curves (CVPR 2025 · similarity 0.83)
- Injecting Frame-Event Complementary Fusion into Diffusion for Optical Flow in Challenging Scenes (NeurIPS 2025 · similarity 0.83)
- DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement (CVPR 2024 · similarity 0.82)
- Disentangled Motion Modeling for Video Frame Interpolation (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.