Research map / Detection and segmentation / 3D detection and driving
Weak-to-Strong 3D Object Detection with X-Ray Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes X-Ray Distillation, which trains a teacher on temporally aggregated Object-Complete LiDAR frames and distills it into a standard-input student for supervised and semi-supervised 3D object detection.
Problem
LiDAR-based 3D object detection suffers from sparsity and occlusion because each frame is a single-viewpoint 2.5D scan; existing fixes rely on extra modules or architecture changes that limit applicability.
Approach
Aggregate points of each object across past and future frames of a sequence (via tracking and point cloud registration) to build Object-Complete frames, train a frozen-then-distilled teacher on them, and distill its backbone, head, and detection outputs into a stronger student that sees only the original frames.
Key results
- On ONCE semi-supervised, X-Ray PT SECOND reaches 68.43 mAP (Small split) vs 67.06 for Proficient Teacher, and X-Ray PT CenterPoint reaches 70.55 mAP (Large split).
- On Waymo supervised, X-Ray DSVT Pillar-Scaled improves L1 mAP from 79.6 to 80.1 and X-Ray SECOND-Scaled improves from 66.8 to 68.3.
- On NuScenes, X-Ray CenterPoint-Voxel improves mAP from 53.4 to 54.3 and NDS from 61.3 to 62.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply X-Ray distillation | SECOND-Scaled · Waymo Open Dataset validation, L1 | mAP: 66.8 -> 68.3 | Helped |
| Apply X-Ray distillation | SECOND · Waymo Open Dataset validation, L1 | mAP: 67.2 -> 67.0 | Hurt |
| Apply X-Ray distillation | CenterPoint · Waymo Open Dataset validation, L1 | mAP: 74.4 -> 73.2 | Hurt |
| Apply X-Ray distillation | CenterPoint-Scaled · Waymo Open Dataset validation, L1 | mAP: 74.1 -> 75.2 | Helped |
| Apply X-Ray distillation | DSVT Pillar-Scaled · Waymo Open Dataset validation, L1 | mAP: 79.6 -> 80.1 | Helped |
| Apply X-Ray distillation | CBGS · NuScenes validation | mAP: 50.0 -> 50.8 | Helped |
| Apply X-Ray distillation | CenterPoint-Voxel · NuScenes validation | NDS: 61.3 -> 62.9 | Helped |
| Add X-Ray Teacher to Proficient Teacher | SECOND · ONCE validation, Small | mAP: 67.06 -> 68.43 (+1.37) | Helped |
| Add X-Ray Teacher to Mean Teacher | CenterPoint · ONCE validation, Small | mAP: 66.47 -> 67.60 (+1.13) | Helped |
| Fine-tune the teacher on Object-Complete frames (ablation) | X-Ray semi-supervised teacher, SECOND · ONCE validation | mAP: X-Ray MT SECOND 60.12 -> X-Ray MT SECOND* (ours) 65.26; X-Ray PT SECOND 61.48 -> X-Ray PT SECOND* (ours) 68.43 | Helped |
| Match BEV features as well as classification and regression heads (ablation) | SECOND on Waymo validation · Waymo Open Dataset validation, L1 | mAP/mAPH: 67.5/63.4 -> 67.8/63.5 | Helped |
| Use the full distillation pipeline instead of BEV and heads matching (ablation) | SECOND-Scaled · Waymo Open Dataset validation, L1 | mAP/mAPH: 67.8/63.5 -> 68.3/64.3 | Helped |
Surprising findings
- Training on Object-Complete frames boosts CenterPoint mAP on NuScenes from 59.2 to 79.5, yet the teacher is only used to distill into a standard-input student.
- The X-Ray Student for SECOND on Waymo L1 mAP (67.0) is slightly below the baseline (67.2), despite improved L2 mAP in some settings being mixed.
Limitations
- Object-Complete frames in the semi-supervised setting are noisier because they rely on predicted boxes and tracking, so reconstruction is imperfect.
- Gains on NuScenes are smaller (about 0.8 to 1 mAP) because NuScenes object-complete frames are less informative than Waymo's.
- Requires sequential LiDAR data and tracking labels or a pretrained detector to build object-complete frames.
Experimental setup
Datasets: NuScenes Waymo Open Dataset ONCE
Benchmarks: NuScenes Waymo Open Dataset ONCE
Metrics: mAP mAPH NDS
Baselines: SECOND CenterPoint CBGS DSVT Transfusion-L Mean Teacher Proficient Teacher Li et al. shape-prior methods
Builds on: Mean Teacher Proficient Teacher OpenPCDet GeDi
Closest papers by meaning
- SimDistill: Simulated Multi-Modal Distillation for BEV 3D Object Detection (AAAI 2024 · similarity 0.88)
- A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection (CVPR 2024 · similarity 0.88)
- VSRD: Instance-Aware Volumetric Silhouette Rendering for Weakly Supervised 3D Object Detection (CVPR 2024 · similarity 0.88)
- Weakly Supervised Monocular 3D Detection with a Single-View Image (CVPR 2024 · similarity 0.87)
- Semi-supervised 3D Object Detection with PatchTeacher and PillarMix (AAAI 2024 · similarity 0.87)
- Beyond Virtual Points: Depth-Enhanced LiDAR-only 3D Object Detection with Semi-Supervised Learning (Student Abstract) (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.