Research map / Detection and segmentation / 3D detection and driving
HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HGSFusion, a radar-camera 3D detector that generates denser radar points using Gaussian and uniform PDFs guided by image masks (accounting for DOA errors) and fuses BEV features via spatial and modality sync.
Problem
3D object detection from sparse 4D radar point clouds with DOA angle estimation errors, and radar-camera fusion degraded by missing image depth and poor image quality under adverse lighting.
Approach
Generate hybrid radar points by sampling from Gaussian PDFs around foreground (image-mask) radar points and uniform PDFs elsewhere to tolerate DOA errors, then use radar-derived spatial patterns to enhance image BEV features and a learned channel-weighting (modality sync) to balance the two modalities.
Key results
- On VoD validation, HGSFusion reaches 58.96 EAA AP and 79.46 RoI AP, surpassing LXL by 2.65% and 6.53%.
- On TJ4DRadSet test, HGSFusion achieves 37.21 3D mAP and 43.23 BEV mAP, exceeding LXL by 0.89% and 2.03%.
- Ablation on VoD: RHGM adds 2.41% EAA AP and DSM adds 1.17% EAA AP over the raw radar+camera baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HGSFusion instead of LXL | Radar-camera 3D detection · VoD validation | EAA AP mAP: 56.31 -> 58.96 | Helped |
| Use HGSFusion instead of LXL | Radar-camera 3D detection · VoD validation | RoI AP mAP: 72.93 -> 79.46 | Helped |
| Use HGSFusion instead of LXL | Radar-camera 3D detection · TJ4DRadSet test | 3D mAP: 36.32 -> 37.21 | Helped |
| Use HGSFusion instead of LXL | Radar-camera 3D detection · TJ4DRadSet test | BEV mAP: 41.20 -> 43.23 | Helped |
| Add RHGM to radar-camera fusion (ablation) | VoD fusion baseline · VoD validation | EAA AP: +2.41% | Helped |
| Add RHGM to radar-camera fusion (ablation) | VoD fusion baseline · VoD validation | RoI AP: +1.56% | Helped |
| Add DSM to radar-camera fusion (ablation) | VoD fusion baseline · VoD validation | EAA AP: +1.17% | Helped |
| Add DSM to radar-camera fusion (ablation) | VoD fusion baseline · VoD validation | RoI AP: +1.18% | Helped |
| Replace hybrid Gaussian-uniform generation with uniform-only generation (ablation) | HGSFusion · VoD validation | EAA AP: 0.82% | Hurt |
| Replace hybrid Gaussian-uniform generation with uniform-only generation (ablation) | HGSFusion · VoD validation | RoI AP: 3.20% | Hurt |
| Replace hybrid Gaussian-uniform generation with Gaussian-only generation (ablation) | HGSFusion · VoD validation | EAA AP: 1.83% | Hurt |
| Replace hybrid Gaussian-uniform generation with Gaussian-only generation (ablation) | HGSFusion · VoD validation | RoI AP: 3.52% | Hurt |
Surprising findings
- Naive radar-camera fusion (Base-R+C) underperforms radar-only in dark scenes on TJ4DRadSet (3D mAP 4.27 vs 13.19), showing image input can hurt under poor lighting.
- Performance drops for the Cyclist category on VoD, attributed to bicycle-like objects (racks, scooters) that are hard to distinguish.
Limitations
- Performance decline on the Cyclist category due to confusable bicycle-like objects in VoD.
- Image backbone is frozen to prevent overfitting, limiting adaptation.
- 2D spatial pattern preferred because radar lacks accurate elevation, limiting height estimation.
Experimental setup
Datasets: VoD TJ4DRadSet
Benchmarks: VoD TJ4DRadSet
Metrics: EAA AP RoI AP 3D AP BEV AP mAP
Baselines: PointPillars RadarPillarNet FUTR3D BEVFusion RCFusion LXL TL-4DRCF RCBEVDet RPFA-Net SMURF
Builds on: BEVFusion PointPillars Mask2Former OpenPCDet
Closest papers by meaning
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.89)
- CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection (NeurIPS 2024 · similarity 0.88)
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.87)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.86)
- RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion (CVPR 2025 · similarity 0.86)
- RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detection (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.