Research map / Label-efficient and robust learning / Recognition and long-tail
RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RC-AutoCalib, an end-to-end online radar-camera extrinsic calibration network that fuses frontal and bird's-eye-view features with cross-attention matching and LiDAR-guided noise-resistant supervision, evaluated on nuScenes.
Problem
Online automatic geometric calibration of 3D radar and camera, which is hard due to sparse radar data and elevation (height) measurement uncertainty; prior radar-camera online methods only handle rotation and no benchmark exists.
Approach
A Dual-Perspective (frontal + BEV) representation with a Selective Fusion Mechanism to pick reliable features from each view, a Multi-Modal Cross-Attention module for radar-image correspondence, explicit feature matching supervision, and a Noise-Resistant Matcher that uses LiDAR points to filter unreliable radar matches during training.
Key results
- Under R1 (±10°, ±0.25m) the method reaches 0.427° mean rotation error and 9.498 cm mean translation error, versus 0.807° and 12.608 cm for CalibDepth.
- Under the larger R2 (±20°, ±1.5m) range it achieves 0.852° rotation and 47.537 cm translation error, better than CalibDepth (1.686°, 55.380 cm).
- Ablation: the Selective Fusion, cross-attention, explicit matching supervision and Noise-Resistant Matcher progressively reduce rotation error from 0.657° to 0.427° in the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RC-AutoCalib instead of CalibDepth | radar-camera calibration · nuScenes R1 | mean rotation error (°): 0.807 -> 0.427 | Helped |
| use RC-AutoCalib instead of NetCalib2 | radar-camera calibration · nuScenes R1 | mean translation error (cm): 12.297 -> 9.498 | Helped |
| use RC-AutoCalib instead of CalibDepth | radar-camera calibration · nuScenes R2 | mean rotation error (°): 1.686 -> 0.852 | Helped |
| use RC-AutoCalib instead of CalibDepth | radar-camera calibration · nuScenes R2 | mean translation error (cm): 55.380 -> 47.537 | Helped |
| add a BEV perspective to FV (ablation) | RC-AutoCalib · nuScenes R1 | mean rotation error (°): 0.657 -> 0.575 | Helped |
| add a BEV perspective to FV (ablation) | RC-AutoCalib · nuScenes R1 | mean translation error (cm): 12.602 -> 12.315 | Helped |
| add Selective Fusion (ablation) | RC-AutoCalib with FV and BEV · nuScenes R1 | mean rotation error (°): 0.575 -> 0.529 | Helped |
| add Multi-Modal Cross-Attention (ablation) | RC-AutoCalib with Selective Fusion · nuScenes R1 | mean rotation error (°): 0.529 -> 0.502 | Helped |
| add Explicit Feature Matching Supervision (ablation) | RC-AutoCalib with Multi-Modal Cross-Attention · nuScenes R1 | mean translation error (cm): 12.574 -> 9.627 | Helped |
| add Noise-Resistant Matcher (ablation) | RC-AutoCalib · nuScenes R1 | mean rotation error (°): 0.463 -> 0.427 | Helped |
| add Noise-Resistant Matcher (ablation) | RC-AutoCalib · nuScenes R1 | mean translation error (cm): 9.627 -> 9.498 | Helped |
| replace Selective Fusion with addition fusion (ablation) | RC-AutoCalib · nuScenes R1 | mean rotation error (°): 0.529 -> 0.642 | Hurt |
Surprising findings
- Explicit matching loss weight beta=0.5 gave the lowest rotation error but the highest translation error, showing a rotation-translation trade-off.
Limitations
- Translation error remains high under the large mis-calibration range (R2), e.g. 47.5 cm mean.
- Radar height ambiguity and sparsity still limit translation accuracy; module gains on translation were inconsistent in ablation.
- Evaluated only on a subset of nuScenes with a single radar-camera setup.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: rotation mean absolute error (degrees) translation mean absolute error (cm)
Baselines: LCCNet NetCalib2 CalibDepth Schöller et al. (radar-camera rotational calibration)
Builds on: CalibDepth SKNet LoFTR DepthAnything ZoeDepth
Closest papers by meaning
- RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection (ICLR 2025 · similarity 0.85)
- Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation (NeurIPS 2026 · similarity 0.84)
- Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions (CVPR 2024 · similarity 0.83)
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.82)
- TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion (CVPR 2025 · similarity 0.82)
- CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.