Research map / Detection and segmentation / 3D detection and driving
Improving Distant 3D Object Detection Using 2D Box Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LR3D, which trains camera-based 3D detectors with only 2D box labels for distant objects by learning an implicit 2D-box-to-depth mapping (IP-Head), plus a new Long-range Detection Score metric.
Problem
Camera-based 3D object detection degrades beyond the range of 3D annotations (~40-70m) because LiDAR sparsity makes distant 3D box labels hard and costly to obtain.
Approach
An Implicit Projection Head predicts per-instance dynamic MLP weights from object features so that a 2D box maps to depth; it is trained on close objects with 3D supervision and then applied to distant objects using only 2D boxes. Projection augmentation generates extra 2D-box/depth pairs from the projection geometry, and a teacher-student pipeline transfers the predictions to BEV detectors as pse
Key results
- On KITTI, IP-FCOS3D raises distant (40m+) LDS from 4.9% to 36.2% without distant 3D labels, close to fully supervised FCOS3D at 38.2%.
- On nuScenes, LR3D improves distant (40-51.2m) LDS by 14.3 points over FCOS3D baseline (16.1 vs 1.8 is different subset; reported 14.3% gain).
- Using LR3D as a teacher, DID-M3D trained without distant 3D labels reaches 39.0% distant LDS vs 38.9% fully supervised on KITTI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add IP-Head with distant 2D box supervision | FCOS3D · KITTI val, distant (40m-Inf) | LDS: 4.9 -> 36.2 | Helped |
| Train with LR3D teacher pseudo-labels | DID-M3D · KITTI val, distant (40m-Inf) | LDS: 24.2 -> 39.0 | Helped |
| Add IP-Head with distant 2D box supervision | FCOS3D · nuScenes val, distant (40m-51.2m) | LDS: 1.8 -> 16.1 | Helped |
| Train with LR3D teacher pseudo-labels | BEVFormer-S · nuScenes val, distant (40m-51.2m) | LDS: 2.4 -> 18.3 | Helped |
| Use dynamic rather than shared f(θ) weights (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 32.5 -> 36.2 | Helped |
| Add Sin. Cos. positional encoding (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 13.3 -> 36.2 | Helped |
| Use both box width and height as descriptors rather than either alone (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 30.6; 34.9; 36.2 | Helped |
| Vary f(θ) from 2 layers to 1, 3, or 4 layers (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 36.2 -> 32.4; 35.2; 34.9 | Hurt |
| Vary f(θ) channel count from 16 to 8, 32, or 64 (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 36.2 -> 34.5; 35.6; 36.1 | Hurt |
| Add projection augmentation (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: 26.5 -> 36.2 | Helped |
| Replace projection augmentation with Copy-Paste augmentation (ablation) | LR3D (IP-FCOS3D) · KITTI val, distant (40m-Inf) | LDS: N/A | Hurt |
Surprising findings
- Copy-paste augmentation to simulate distant samples causes a large performance drop, unlike projection augmentation.
- Existing mAP with fixed thresholds yields near-zero scores for distant objects (e.g., 0.43% mAP for DID-M3D beyond 40m), motivating the relative-distance LDS metric.
Limitations
- Relies on 3D annotations for close objects and cannot fully remove 3D labeling needs for the detector pipeline.
- Evaluation of distant objects is limited to 2D-supervised settings with dataset-specific 40m split; extremely distant results are only qualitative on KITTI DontCare objects.
Experimental setup
Datasets: KITTI nuScenes Cityscapes3D Waymo Open Dataset Argoverse 2
Benchmarks: KITTI nuScenes
Metrics: Long-range Detection Score (LDS) mAP mATE mASE mAOE
Baselines: FCOS3D ImVoxelNet CaDDN MonoFlex GUPNet DID-M3D BEVFormer
Builds on: FCOS3D FastRCNN BEVFormer nuScenes Detection Score
Closest papers by meaning
- LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels (CVPR 2024 · similarity 0.87)
- VSRD: Instance-Aware Volumetric Silhouette Rendering for Weakly Supervised 3D Object Detection (CVPR 2024 · similarity 0.87)
- Beyond Virtual Points: Depth-Enhanced LiDAR-only 3D Object Detection with Semi-Supervised Learning (Student Abstract) (AAAI 2025 · similarity 0.87)
- Far3D: Expanding the Horizon for Surround-View 3D Object Detection (AAAI 2024 · similarity 0.86)
- NBA3D: Neighbor-Based Confidence Adjustment for 3D Rare Object Detection Using LiDAR (AAAI 2025 · similarity 0.86)
- Weakly Supervised Monocular 3D Detection with a Single-View Image (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.