Research map / Detection and segmentation / 3D detection and driving
CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRKD, a BEV-space cross-modality knowledge distillation framework that transfers knowledge from a LiDAR-Camera teacher to a Camera-Radar 3D object detection student on nuScenes.
Problem
Camera-Radar (CR) 3D object detectors lag behind LiDAR-Camera (LC) detectors, and no prior KD method supports a fusion-to-fusion (LC-to-CR) distillation path that handles the radar domain gap.
Approach
Distill from a BEVFusion LC teacher to a gated BEVFusion CR student using four losses tailored to radar and camera: cross-stage radar distillation against the teacher's objectness heatmap, mask-scaling feature distillation aware of range and velocity, relation distillation on cosine affinity matrices, and class-weighted response distillation that emphasizes dynamic classes.
Key results
- CRKD improves the student's mAP and NDS by 3.5% and 3.2% on nuScenes val (student BEVFusion-CR* 43.2 to 46.7 mAP with SwinT; NDS 54.1 to 57.3 for baseline vs CRKD).
- On nuScenes test, CRKD with SwinT reaches 48.7 mAP and 58.7 NDS using single-frame image input without test-time optimization.
- Response distillation gives the largest single-module gain in ablations (mAP 43.2 to 44.9 with gated fusion and RespD alone), and per-class gains are largest for dynamic classes such as truck (+6.3 AP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill from an LC teacher to a CR student with CRKD | BEVFusion-CR* · nuScenes val | mAP: 43.2 -> 46.7 | Helped |
| add a gated network for adaptive sensor fusion (ablation) | BEVFusion-CR · nuScenes val | mAP: 43.2 -> 44.9 | Helped |
| add response distillation (ablation) | Gated BEVFusion-CR · nuScenes val | mAP: 44.9 -> 45.7 | Helped |
| add cross-stage radar distillation (CSRD) (ablation) | Gated BEVFusion-CR with RespD · nuScenes val | mAP: 45.7 -> 46.0 | Helped |
| add mask-scaling feature distillation (MSFD) (ablation) | Gated BEVFusion-CR with RespD and CSRD · nuScenes val | mAP: 46.0 -> 46.2 | Helped |
| add relation distillation (RelD) (ablation) | Gated BEVFusion-CR with RespD, CSRD, and MSFD · nuScenes val | mAP: 46.2 -> 46.7 | Helped |
| use teacher heatmap rather than LiDAR feature maps for radar guidance (ablation) | CRKD radar distillation · nuScenes val | mAP: 44.9 -> 46.0 | Helped |
| scale the foreground mask using object range and velocity (ablation) | CRKD feature distillation · nuScenes val | mAP: 46.0 -> 46.2 | Helped |
| add a convolutional block after downsampling for relation distillation (ablation) | CRKD RelD · nuScenes val | mAP: 45.9 -> 46.2 | Helped |
| weight dynamic classes more heavily in response distillation (ablation) | CRKD RespD · nuScenes val | mAP: 45.3 -> 45.7 | Helped |
| reduce fusion channels to 64 + 64 input and 64 output (ablation) | BEVFusion-CR convolutional fuser · nuScenes val | mAP: 43.2 -> 44.2 | Helped |
| use 128 + 128 input channels and 256 output channels (ablation) | BEVFusion-CR convolutional fuser · nuScenes val | mAP: 43.2 -> 44.4 | Helped |
Surprising findings
- The CR student with CRKD outperforms the LC teacher on some scenes by detecting a missed car and rejecting false predictions, per qualitative results.
- Best standalone design for some KD modules did not necessarily work best when combined with response distillation, which the authors attribute to large LC-to-CR domain discrepancy.
Limitations
- LiDAR is required during training for the teacher, so the approach needs LC data availability.
- Evaluated only on nuScenes with single-frame image input.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes val nuScenes test
Metrics: mAP NDS mATE mASE mAOE mAVE mAAE
Baselines: BEVFusion-CR BEVFusion-LC CenterFusion CRAFT RCBEV RCM-Fusion X3KD UniDistill DistillBEV BEVDistill
Builds on: BEVFusion MonoDistill CMKD CenterPoint BEVDet
Closest papers by meaning
- SCKD: Semi-Supervised Cross-Modality Knowledge Distillation for 4D Radar Object Detection (AAAI 2025 · similarity 0.91)
- RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features (CVPR 2024 · similarity 0.90)
- CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection (NeurIPS 2024 · similarity 0.88)
- Availability-aware Sensor Fusion via Unified Canonical Space (NeurIPS 2025 · similarity 0.87)
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.86)
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.