Research map / Detection and segmentation / 3D detection and driving
RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RCBEVDet, a radar-camera fusion 3D object detector that builds BEV features with a dual-stream radar backbone (RadarBEVNet, with RCS-aware scattering) and fuses them with camera BEV features via deformable cross-attention (CAMF).
Problem
Camera-only 3D object detection lacks accurate depth and robustness in adverse conditions, and existing radar-camera BEV fusion methods use LiDAR-designed radar encoders and suffer from spatial misalignment between radar and camera BEV features.
Approach
A dual-stream radar encoder (PointNet-style point branch plus distance-modulated transformer branch with injection/extraction cross-attention), an RCS-aware BEV scattering step that uses radar cross-section as an object size prior, and a deformable cross-attention plus channel/spatial fusion module to align and fuse radar and camera BEV features.
Key results
- On nuScenes val with ResNet-50 at 256x704, RCBEVDet reaches 56.8 NDS and 45.3 mAP, versus 56.0 NDS for CRN, and 21.3 FPS.
- On nuScenes test with V2-99, RCBEVDet achieves 63.9 NDS, surpassing CRN (ConvNeXt-B, 62.4 NDS).
- On VoD validation region of interest, RCBEVDet reaches 69.80 mAP, versus 69.23 for RCFusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal features (ablation) | BEVDepth · nuScenes val | NDS: 47.5 -> 51.9 (+4.4) | Helped |
| Add temporal features (ablation) | BEVDepth · nuScenes val | mAP: 35.1 -> 40.5 (+5.4) | Helped |
| Add PointPillar radar backbone and BEVFusion (ablation) | BEVDepth + Temporal · nuScenes val | NDS: +1.7 | Helped |
| Replace PointPillar with RadarBEVNet (ablation) | PointPillar + BEVFusion · nuScenes val | mAP: +3.0 | Helped |
| Add CAMF (ablation) | RadarBEVNet model · nuScenes val | NDS: +0.7 | Helped |
| Add Temporal Supervision (ablation) | RCBEVDet · nuScenes val | mAP: 45.6 -> 45.3 (↓0.3) | Hurt |
| Add RCS-aware BEV encoder (ablation) | Point-based radar backbone · nuScenes val | NDS: +1.4 | Helped |
| Add Transformer backbone (ablation) | RadarBEVNet with Point-based backbone and RCS-aware BEV encoder · nuScenes val | NDS: +0.1 | No clear effect |
| Add Injection and Extraction module (ablation) | Dual-stream radar backbone · nuScenes val | NDS: +0.6 | Helped |
| Add Deformable Cross-Attention (ablation) | BEVFusion fusion · nuScenes val | NDS: +0.4 | Helped |
| Add Channel and Spatial Fusion (ablation) | CAMF with Deformable Cross-Attention · nuScenes val | NDS: +0.3 | Helped |
| Use RCBEVDet instead of RCFusion | Radar-camera detection on VoD · VoD validation set, entire annotated area | mAP: 49.65 -> 49.99 | Helped |
Surprising findings
- Directly adding the transformer-based radar backbone yields only marginal gain (+0.1 NDS), which the authors attribute to unmerged feature representations between the two radar branches.
- Temporal supervision improves NDS by 0.4 but decreases mAP by 0.3 on nuScenes val.
Limitations
- Camera stream and radar stream are trained in two stages with the camera stream frozen in the second stage.
- On VoD entire-area evaluation, RCBEVDet's Car AP (40.63) is below RCFusion (41.70).
Experimental setup
Datasets: nuScenes VoD
Benchmarks: nuScenes VoD
Metrics: NDS mAP mATE mASE mAOE mAVE mAAE FPS AP
Baselines: CenterFusion CRAFT RCBEV4d CRN BEVDet BEVDepth SOLOFusion StreamPETR BEVFusion RCFusion PointPillar BEVStereo
Builds on: BEVDepth BEVFusion CRN BEVDet4D PointNet
Closest papers by meaning
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.92)
- CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection (NeurIPS 2024 · similarity 0.91)
- RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion (CVPR 2025 · similarity 0.89)
- HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection (AAAI 2025 · similarity 0.89)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.88)
- CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.