Research map / Detection and segmentation / 3D detection and driving
RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RICCARDO, which predicts object-centric radar hit distributions from monocular detections, convolves them with measured radar points to score range candidates, and rescores them to refine camera-radar 3D detection on nuScenes.
Problem
Radar-camera 3D object detection is hard because radar hits on objects are sparse and dispersed, depending on category, size and orientation, and existing fusion methods learn this relationship implicitly in black-box networks.
Approach
Explicitly model the radar hit distribution conditioned on monocular detection attributes (category, size, orientation, range, azimuth), then use that predicted distribution as a convolution kernel over actual radar points to refine object range estimates.
Key results
- On the nuScenes test set, SparseBEV + RICCARDO reaches NDS 0.695 and mAP 0.630, the best NDS among the compared camera-radar fusion methods (HVDetFusion 0.674).
- On the nuScenes validation set, SparseBEV + RICCARDO improves NDS from 0.592 to 0.622 and mAP from 0.501 to 0.544 over its monocular SparseBEV counterpart.
- Stage 3 reduces mean range error to 0.57 m for class-mean versus 0.83 m for monocular, and the RIC distribution gives MAE 0.47 m versus 0.77 m for an L-shaped baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RICCARDO radar fusion to SparseBEV | SparseBEV (V2-99) · nuScenes test | mAP: 0.603 -> 0.630 | Helped |
| add RICCARDO radar fusion to SparseBEV | SparseBEV (V2-99) · nuScenes test | mATE: 0.425 -> 0.363 | Helped |
| add RICCARDO radar fusion to SparseBEV | SparseBEV (V2-99) · nuScenes test | NDS: 0.675 -> 0.695 | Helped |
| add RICCARDO radar fusion to SparseBEV | SparseBEV (ResNet101) · nuScenes validation | mAP: 0.501 -> 0.544 | Helped |
| add RICCARDO radar fusion to SparseBEV | SparseBEV (ResNet101) · nuScenes validation | mATE: 0.562 -> 0.481 | Helped |
| add RICCARDO radar fusion to SparseBEV | SparseBEV (ResNet101) · nuScenes validation | NDS: 0.592 -> 0.622 | Helped |
| use Stage 2 range matching instead of monocular range estimation | SparseBEV range estimation · nuScenes validation, class-mean | median absolute range error (meter): 0.57 -> 0.52 | Helped |
| use Stage 2 range matching instead of monocular range estimation | SparseBEV range estimation · nuScenes validation, class-mean | mean absolute range error (meter): 0.83 -> 0.94 | Hurt |
| add Stage 3 candidate refinement to Stage 2 | RICCARDO range estimation · nuScenes validation, class-mean | mean/median absolute range error (meter): 0.94/0.52 -> 0.65/0.36 | Helped |
| replace the learned RIC with a uniform distribution within object boundaries (ablation) | RICCARDO Stage 1 distribution · nuScenes validation | range MAE (m): 0.47 -> 0.67 | Hurt |
| replace the learned RIC with a uniform distribution within object boundaries (ablation) | RICCARDO Stage 1 distribution · nuScenes validation | MMS: 0.105 -> 0.078 | Hurt |
| replace the learned RIC with an L-shaped distribution (ablation) | RICCARDO Stage 1 distribution · nuScenes validation | range MAE (m): 0.47 -> 0.77 | Hurt |
Surprising findings
- Simple Stage-2 range extraction has lower median error than monocular but suffers from outliers with larger mean error, so the learned Stage-3 rescoring is needed.
Limitations
- Detection-level fusion only uses high-level monocular detection outputs, so objects missed by the monocular detector cannot be recovered.
- BEV representation cannot separate radar hits from vertically stacked targets, such as a person on a bicycle.
- Search is restricted to 1D radial offset, assuming tangential position, size and orientation from the monocular detector are accurate.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: NDS mAP mATE mASE mAOE mAVE mAAE mean absolute range error mean matching score
Baselines: PGD SparseBEV MVFusion CRN RCBEVDet HyDRa HVDetFusion
Builds on: SparseBEV MMDetection3D CRN MVFusion
Closest papers by meaning
- HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection (AAAI 2025 · similarity 0.86)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.85)
- CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection (NeurIPS 2024 · similarity 0.85)
- RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion (CVPR 2025 · similarity 0.84)
- RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features (CVPR 2024 · similarity 0.84)
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.