Research map / 3D vision / Depth and stereo
TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TacoDepth, a one-stage radar-camera metric depth estimator using a graph-based radar structure extractor and pyramid-based radar fusion with Radar-centered flash attention, supporting both independent and plug-in inference.
Problem
Radar-camera depth estimation is inefficient and non-robust because sparse, noisy radar returns force multi-stage pipelines that rely on intermediate quasi-dense depth.
Approach
Replace multi-stage quasi-dense depth pipelines with a single-stage model: a GNN over radar points captures graph structure, and hierarchical Radar-centered flash attention fuses radar and image features per pyramid layer, with an optional auxiliary branch for initial relative depth.
Key results
- Independent TacoDepth reaches 26.7 ms runtime (about 37.5 fps) on nuScenes 0-50m with MAE 1423.6 mm, versus CaFNet's 103.9 ms and MAE 1674.2 mm.
- Plug-in TacoDepth reduces MAE by 11.7% and RMSE by 13.9% relative to RadarCam-Depth on nuScenes while cutting runtime by 91.8% (29.3 ms vs 358.3 ms).
- On ZJU-4DRadarCam (0-70m), plug-in TacoDepth achieves MAE 983.1 mm vs 1157.0 mm for RadarCam-Depth using DPT-Hybrid.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use one-stage fusion instead of RadarCam-Depth's multi-stage plug-in framework | TacoDepth with DPT-Hybrid · nuScenes test, 0–70m | MAE: 1587.9 -> 1347.1 | Helped |
| use one-stage fusion instead of RadarCam-Depth's multi-stage plug-in framework | TacoDepth with DPT-Hybrid · nuScenes test | inference time: 358.3 -> 29.3 | Helped |
| replace the graph-based Radar structure extractor with an MLP (ablation) | TacoDepth · nuScenes, 0–70m | MAE: 1712.6 -> 1815.6 | Hurt |
| replace the graph-based Radar structure extractor with an MLP (ablation) | TacoDepth · nuScenes, 0–70m | RMSE: 3960.5 -> 4189.8 | Hurt |
| add pyramid-based Radar fusion (ablation) | Radar-Camera depth estimation using Singh et al.'s baseline · nuScenes, 0–70m | MAE: 2073.2 -> 1815.6 | Helped |
| add pyramid-based Radar fusion (ablation) | Radar-Camera depth estimation using Singh et al.'s baseline · nuScenes, 0–70m | RMSE: 4590.7 -> 4189.8 | Helped |
| use L=3 instead of L=1 or L=2 (ablation) | TacoDepth graph extractor and fusion module · nuScenes, 0–70m | MAE: 1840.6 -> 1712.6; 1787.8 -> 1712.6 | Helped |
| increase the layer number from L=3 to L=4 (ablation) | TacoDepth graph extractor and fusion module · nuScenes, 0–70m | MAE: 1712.6 -> 1729.3 | Hurt |
Surprising findings
- Ablation shows the pyramid-based fusion alone (without graph extractor) accounts for most of the MAE gain over Singh et al., with the graph extractor adding a further 5.7% MAE reduction.
Limitations
- Authors state they provide only one efficient implementation of TacoDepth; other techniques are left unexplored.
- Evaluation limited to two datasets (nuScenes with 3D radar, ZJU-4DRadarCam with 4D radar).
Experimental setup
Datasets: nuScenes ZJU-4DRadarCam
Benchmarks: nuScenes ZJU-4DRadarCam
Metrics: MAE RMSE iMAE iRMSE δ1 Rel runtime FLOPs parameters
Baselines: Lin et al. RC-PDA R4Dyn DORN Singh et al. CaFNet Li et al. (Disruption-Compensation) RadarCam-Depth
Builds on: PCA-GM Singh et al. gated fusion radar-camera depth RadarCam-Depth FlashAttention
Closest papers by meaning
- Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation (NeurIPS 2026 · similarity 0.89)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.86)
- HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection (AAAI 2025 · similarity 0.85)
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.83)
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.83)
- RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detection (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.