Research map / Detection and segmentation / 3D detection and driving
GAFusion: Adaptive Fusing LiDAR and Camera with Multiple Guidance for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAFusion, a BEV-based LiDAR-camera 3D detector that uses LiDAR guidance (sparse depth and occupancy) to improve camera depth features, plus multi-scale dual-path transformer and LiDAR-guided adaptive fusion transformer, reaching 73.6% mAP / 74.9% NDS on nuScenes test.
Problem
LiDAR-camera 3D object detection in BEV representations, where camera BEV features lack accurate depth/geometric information and the two modalities interact weakly before fusion.
Approach
Use LiDAR to guide camera features before view transformation (sparse depth guidance and LiDAR occupancy guidance), enlarge receptive fields with additional sparse downsampling and a multi-scale dual-path transformer, and fuse BEV features with a LiDAR-guided adaptive transformer that weights camera features by LiDAR-derived gates.
Key results
- On nuScenes test, GAFusion reaches 73.6% mAP and 74.9% NDS without TTA or ensemble, surpassing MSMDFusion (71.5/74.0) and CMT (70.4/73.0).
- On nuScenes val, GAFusion achieves 72.1% mAP and 73.5% NDS, versus 68.5/71.4 for BEVFusion.
- In ablation on BEVFusion, adding SDG and LOG together raises val mAP from 68.52 to 69.93 and NDS from 71.38 to 72.24.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add additional downsampling and sparse height compression (ablation) | TransFusion · nuScenes test | mAP: 68.9 -> 69.9 | Helped |
| Add additional downsampling and sparse height compression (ablation) | BEVFusion · nuScenes test | mAP: 70.2 -> 71.0 | Helped |
| Add sparse depth guidance and LiDAR occupancy guidance (ablation) | BEVFusion · nuScenes val | mAP: 68.52 -> 69.93 | Helped |
| Add sparse depth guidance (ablation) | BEVFusion · nuScenes val | mAP: 68.52 -> 69.33 | Helped |
| Add LiDAR occupancy guidance (ablation) | BEVFusion · nuScenes val | mAP: 68.52 -> 69.49 | Helped |
| Use LGFT instead of addition fusion (ablation) | BEV feature fusion · nuScenes val | mAP: 70.85 -> 71.63 | Helped |
| Use LGFT instead of concatenation fusion (ablation) | BEV feature fusion · nuScenes val | mAP: 70.92 -> 71.63 | Helped |
| Add the adaptive mechanism to LGFT (ablation) | BEV feature fusion · nuScenes val | mAP: 71.63 -> 71.79 | Helped |
| Add temporal fusion (ablation) | Addition fusion · nuScenes val | mAP: 70.85 -> 71.17 | Helped |
| Add temporal fusion (ablation) | Concatenation fusion · nuScenes val | mAP: 70.92 -> 71.21 | Helped |
| Add temporal fusion (ablation) | LGFT · nuScenes val | mAP: 71.63 -> 71.91 | Helped |
| Add temporal fusion (ablation) | LGAFT · nuScenes val | mAP: 71.79 -> 72.08 | Helped |
Surprising findings
- LOG alone gives a larger gain than SDG alone (1.0% vs 0.8% mAP), which the authors attribute to direct 3D feature interaction providing located information.
- Adding the additional downsampling and sparse height compression to TransFusion and BEVFusion yields gains of only about 0.5-1.0% mAP, a modest effect.
Limitations
- Temporal fusion yields only marginal gains (about 0.3% mAP) using only the previous frame; multi-frame extension is left to future work.
- Evaluation is restricted to nuScenes, with no reported results on other driving datasets.
- Reported test-set numbers differ from the introduction's claims (72.1% mAP / 73.5% NDS in the intro refer to validation results).
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes test nuScenes val
Metrics: mAP NDS mATE mASE mAOE mAVE mAAE
Baselines: BEVFusion TransFusion CMT MSMDFusion DeepInteraction SparseFusion FocalFormer3D BEVFormer PETRv2
Builds on: BEVFusion TransFusion BEVDet4D VoxelNet Swin Transformer
Closest papers by meaning
- DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection (CVPR 2022 · similarity 0.90)
- BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (NeurIPS 2022 · similarity 0.90)
- BEVDilation: LiDAR-Centric Multi-Modal Fusion for 3D Object Detection (AAAI 2026 · similarity 0.89)
- CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detection (CVPR 2026 · similarity 0.89)
- SimDistill: Simulated Multi-Modal Distillation for BEV 3D Object Detection (AAAI 2024 · similarity 0.89)
- IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.