Research map / Detection and segmentation / 3D detection and driving
RETR: Multi-View Radar Detection Transformer for Indoor Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends DETR to multi-view (horizontal and vertical) indoor radar heatmaps for 2D object detection and instance segmentation, adding tunable positional encoding, a tri-plane loss, and a learnable radar-to-camera transform.
Problem
Indoor radar perception from multi-view radar heatmaps lacks methods that exploit the shared depth axis and radar-to-camera geometry; existing pipelines like RFMask use only the horizontal view for box loss and fixed-height vertical proposals.
Approach
A DETR-based detector that takes horizontal and vertical radar heatmaps, uses top-K feature selection and a tunable positional encoding that splits the positional dimension between depth and angle, predicts 3D boxes in radar coordinates supervised on three planes (horizontal radar, vertical radar, image), and learns the radar-to-camera rotation via an so(3) exponential reparameterization.
Key results
- On MMVR P2S1, RETR reaches 46.75 AP vs 31.37 for RFMask (+15.38 AP).
- On HIBER WALK, RETR reaches 22.09 AP vs 17.77 for RFMask (+4.32 AP).
- On MMVR P2S1 segmentation, RETR achieves 77.21 IoU vs 65.30 for RFMask (+11.91 IoU).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Top-K selection of horizontal and vertical radar-view features | DETR · MMVR P2S1 | AP: 31.37 -> 39.71 | Helped |
| Add TPE at the decoder and tri-plane BBox loss | DETR (Top-K) · MMVR P2S1 | AP: +6.23 | Helped |
| Add TPE at the encoder | RETR (TPE@Dec.) · MMVR P2S1 | AP: 45.94 -> 46.75 | Helped |
| Add full RETR modifications | RFMask · MMVR P2S1 | AP: +15.38 | Helped |
| Add full RETR modifications | RFMask · HIBER WALK | AP: 17.77 -> 22.09 | Helped |
| Add full RETR modifications | RFMask · MMVR P2S1 | IoU: 11.77 | Helped |
| Use TPE at both encoder and decoder instead of decoder-only TPE (ablation) | RETR (TPE@Dec.) · MMVR P2S1 | AP: 45.94 -> 46.75 | Helped |
| Set TPE depth-dimension ratio α to 0.2, 0.4, or 0.8 instead of 0.6 (ablation) | RETR · MMVR P2S1 | AP: 46.75; 44.35; 45.16; 43.81 | Mixed |
| Add learnable radar-to-camera transformation instead of relying on calibration (ablation) | RETR · MMVR P2S1 | AP: 42.97 -> 46.75 | Helped |
| Add vertical-plane BBox supervision to the H+I loss (ablation) | RETR · MMVR P2S1 | AP: 42.18 -> 46.75 | Helped |
| Vary Top-K feature count from 64 or 100 to 196 or 256 (ablation) | RETR · MMVR P2S1 | AP: 38.39; 38.82; 46.97; 46.75 | Mixed |
| Reduce training data to 0.1× or 0.5× instead of 1.0× (ablation) | RETR · MMVR P2S1 | AP: 46.75 -> 37.10; 40.84 | Hurt |
Surprising findings
- Plain DETR with only the horizontal radar view underperforms RFMask (29.38 vs 31.37 AP on MMVR P2S1).
Limitations
- Arm positions and regions with weak radar reflections are hard to predict.
- Multi-path ghost targets from ground, ceiling, and metal are not handled.
- Performance drops substantially on the unseen environment in MMVR P2S2.
Experimental setup
Datasets: MMVR HIBER
Benchmarks: MMVR P2S1 MMVR P2S2 HIBER WALK HIBER MULTI
Metrics: AP AP50 AP75 AR1 AR10 IoU
Baselines: RFMask DETR DETR (Top-K) RETR (TPE@Dec.)
Builds on: DETR Conditional DETR RFMask GIoU loss
Closest papers by meaning
- RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion (CVPR 2025 · similarity 0.86)
- RAPTR: Radar-based 3D Pose Estimation using Transformer (NeurIPS 2025 · similarity 0.85)
- SIRA: Scalable Inter-frame Relation and Association for Radar Perception (CVPR 2024 · similarity 0.84)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.84)
- V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception (NeurIPS 2025 · similarity 0.83)
- RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.