Research map / 3D vision / 3D generation
Weakly Supervised Monocular 3D Detection with a Single-View Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SKD-WM3D, a weakly supervised monocular 3D detector that uses only single-view images and off-the-shelf depth pseudo-labels, distilling depth-fused 3D-like knowledge from a self-teaching network into a lightweight monocular detector.
Problem
Weakly supervised monocular 3D detection currently needs extra data such as LiDAR point clouds or multi-view/multi-frame images, which limits applicability; the depth ambiguity of single-view images also hurts localization.
Approach
A self-knowledge distillation framework in which a depth-guided self-teaching network (fusing depth features with image features via cross/self-attention) teaches a plain monocular detector through soft-label distillation, combined with an uncertainty-aware distillation loss and a gradient-targeted transfer modulation that balances learning pace between the two networks.
Key results
- On KITTI test Car (IoU 0.7), SKD-WM3D achieves AP3D|R40 of 8.95 / 5.54 / 4.53 (easy/moderate/hard), above WeakMono3D (6.98 / 4.85 / 4.45).
- On KITTI val Car (IoU 0.5), SKD-WM3D reaches AP3D|R40 of 50.21 easy, 41.57 moderate, 36.92 hard, versus WeakM3D's 50.16 / 29.94 / 23.11.
- Inference runs at 30.3 FPS versus 13.9 FPS for WeakM3D, and nuScenes AP improves from 0.214 to 0.242.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SKD-WM3D instead of WeakMono3D | Weakly supervised monocular 3D detection · KITTI test Car | AP3D|R40: 4.85 -> 5.54; 4.45 -> 4.53 | Helped |
| Use SKD-WM3D instead of WeakMono3D | Weakly supervised monocular 3D detection · KITTI val Car | AP3D|R40: 49.37 -> 50.21; 39.01 -> 41.57; 36.34 -> 36.92 | Helped |
| Use SKD-WM3D instead of WeakM3D | Weakly supervised monocular 3D detection · nuScenes val Car | AP: 0.214 -> 0.242 | Helped |
| Use SKD-WM3D instead of WeakM3D | Weakly supervised monocular 3D detection · nuScenes val Car | ATE: 0.814 -> 0.795 | Helped |
| Use SKD-WM3D instead of WeakM3D | Weakly supervised monocular 3D detection · nuScenes val Car | ASE: 0.234 -> 0.231 | Helped |
| Use SKD-WM3D instead of WeakM3D | Weakly supervised monocular 3D detection · nuScenes val Car | AAE: 0.682 -> 0.659 | Helped |
| Add self-knowledge distillation using MDN and DSN instead of training MDN alone (ablation) | SKD-WM3D · KITTI val Car Easy, Moderate, and Hard | AP3D|R40: 0.00 -> 50.21; 0.00 -> 41.57; 0.00 -> 36.92 | Helped |
| Add MDN and self-knowledge distillation to DSN-only training (ablation) | SKD-WM3D · KITTI val Car Easy, Moderate, and Hard | AP3D|R40: 40.96 -> 50.21; 31.02 -> 41.57; 26.27 -> 36.92 | Helped |
| Add the uncertainty-aware distillation loss (ablation) | SKD-WM3D · KITTI val Car Easy, Moderate, and Hard | AP3D|R40: 44.61 -> 48.13; 35.74 -> 39.02; 34.82 -> 35.70 | Helped |
| Add the gradient-targeted transfer modulation strategy (ablation) | SKD-WM3D · KITTI val Car Easy, Moderate, and Hard | AP3D|R40: 44.61 -> 46.30; 35.74 -> 38.91; 34.82 -> 35.44 | Helped |
| Add both the uncertainty-aware distillation loss and gradient-targeted transfer modulation (ablation) | SKD-WM3D · KITTI val Car Easy, Moderate, and Hard | AP3D|R40: 44.61 -> 50.21; 35.74 -> 41.57; 34.82 -> 36.92 | Helped |
Surprising findings
- Training the monocular detection network alone (without the depth-guided teacher) yields essentially no meaningful detections (0.00 AP), showing the depth-ambiguity problem is severe without depth information.
Limitations
- Performance on heavily occluded or distant objects drops in orientation and depth estimation.
- Still lags fully supervised methods on KITTI val (e.g., MonoDETR AP3D 43.48 moderate).
Experimental setup
Datasets: KITTI 3D nuScenes
Benchmarks: KITTI 3D test KITTI 3D val nuScenes val
Metrics: AP_BEV|R40 AP_3D|R40 AP ATE ASE AAE FPS
Baselines: WeakM3D WeakMono3D VS3D Autolabels MonoDistill CenterNet MonoGRNet M3D-RPN MonoPair MonoDLE GUPNet Kinematic
Builds on: WeakM3D WeakMono3D CenterNet DLA
Closest papers by meaning
- FD3D: Exploiting Foreground Depth Map for Feature-Supervised Monocular 3D Object Detection (AAAI 2024 · similarity 0.89)
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders (NeurIPS 2024 · similarity 0.88)
- Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection (AAAI 2024 · similarity 0.88)
- MonoCD: Monocular 3D Object Detection with Complementary Depths (CVPR 2024 · similarity 0.88)
- VSRD: Instance-Aware Volumetric Silhouette Rendering for Weakly Supervised 3D Object Detection (CVPR 2024 · similarity 0.87)
- MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.