Research map / Detection and segmentation / 3D detection and driving
CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRT-Fusion, a radar-camera 3D detector that fuses radar and camera in both perspective and BEV views, then estimates per-pixel velocity and occupancy to motion-compensate and recurrently fuse multi-frame BEV features.
Problem
Radar-camera 3D object detection methods that add temporal information concatenate BEV features across frames without accounting for object motion, which degrades accuracy on dynamic objects.
Approach
Radar features enhance camera perspective-view features (RCA) before depth prediction; a Motion Feature Estimator predicts per-pixel BEV velocity and occupancy, and Motion Guided Temporal Fusion shifts previous BEV features by the predicted velocity and fuses them recurrently with occupancy gating.
Key results
- On nuScenes validation with ResNet-50, CRT-Fusion reaches 57.2 NDS and 50.0 mAP in the ablation setting, up from 55.4 NDS / 47.8 mAP for BEV fusion alone.
- Abstract claims +1.7% NDS and +1.4% mAP over the previous best radar-camera method on nuScenes.
- With ResNet-50 and CBGS, CRT-Fusion outperforms RCBEVDet by 2.9% NDS and 5.5% mAP on the validation set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add radar-camera BEV fusion with gating (ablation) | BEVDepth baseline · nuScenes val | NDS: 47.4 -> 55.4 | Helped |
| Add radar-camera BEV fusion with gating (ablation) | BEVDepth baseline · nuScenes val | mAP: 37.8 -> 47.8 | Helped |
| Add Radar-Camera Azimuth attention (ablation) | BEVDepth with BEV Fusion · nuScenes val | NDS: 55.4 -> 56.1 | Helped |
| Add Radar-Camera Azimuth attention (ablation) | BEVDepth with BEV Fusion · nuScenes val | mAP: 47.8 -> 48.9 | Helped |
| Add MFE and MGTF (ablation) | CRT-Fusion with BEV Fusion and RCA · nuScenes val | NDS: 56.1 -> 57.2 | Helped |
| Add MFE and MGTF (ablation) | CRT-Fusion with BEV Fusion and RCA · nuScenes val | mAP: 48.9 -> 50.0 | Helped |
| Enable MFE and MGTF (ablation) | Camera-only BEVDepth · nuScenes val | NDS: 47.4 -> 46.9 | Hurt |
| Enable MFE and MGTF (ablation) | Camera-only BEVDepth · nuScenes val | mAP: 37.8 -> 37.3 | Hurt |
| Use 6 previous frames instead of 0 (ablation) | CRT-Fusion · nuScenes val | NDS: 50.4 -> 57.2 | Helped |
| Use 6 previous frames instead of 0 (ablation) | CRT-Fusion · nuScenes val | mAP: 44.4 -> 50.0 | Helped |
| Use 7 rather than 6 previous frames (ablation) | CRT-Fusion · nuScenes val | mAP: 50.0 -> 49.8 | Hurt |
| Use CRT-Fusion instead of CRN | ResNet-101 configuration · nuScenes val, Sunny | mAP: 54.8 -> 54.7 | Hurt |
Surprising findings
- Applying the MFE and MGTF modules to the camera-only BEVDepth baseline slightly degrades performance (NDS 47.4 to 46.9, mAP 37.8 to 37.3), which the authors attribute to difficulty estimating velocity from camera alone.
Limitations
- Computational cost grows with the number of previous frames in the parallel fusion structure, limiting how many frames can be used given hardware constraints.
- Velocity estimation from camera alone is unreliable, so the motion modules depend on radar input and do not help camera-only models.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: NDS mAP mATE mAOE mAVE
Baselines: BEVDepth CRN RCBEVDet RCM-Fusion RADIANT CenterFusion CRAFT CRKD RGVT RVT
Builds on: BEVDepth CenterPoint CRN SA-BEV PointPillars
Closest papers by meaning
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.91)
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.90)
- RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion (CVPR 2025 · similarity 0.89)
- HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection (AAAI 2025 · similarity 0.88)
- CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation (CVPR 2024 · similarity 0.88)
- IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.