Research map / Detection and segmentation / 3D detection and driving
Learning Occupancy for Monocular 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OccupancyM3D, which learns voxel occupancy in both frustum and regular 3D space from ray-traced LiDAR-derived labels to enhance features for monocular 3D detection.
Problem
Monocular 3D object detection lacks accurate 3D information, and existing methods focus on geometry constraints and dense depth rather than learning discriminative 3D features in frustum and 3D space.
Approach
Generate free/occupied/unknown voxel occupancy labels from synchronized sparse LiDAR via ray tracing, supervise occupancy in frustum space and 3D space with focal loss, and use the predicted occupancy to reweight the features before detection (built on CaDDN).
Key results
- On KITTI test Car, AP3D moderate 17.02 vs. 13.41 for CaDDN (R40, IoU 0.7).
- On Waymo val Vehicle LEVEL 1, 3D mAP at IoU 0.5 is 28.99 vs. 17.54 for CaDDN.
- Ablation on KITTI val Car AP3D (moderate) rises from 17.05 baseline to 19.96 with occupancy learning in both frustum and 3D space.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add occupancy learning in frustum space (ablation) | CaDDN-based detector · KITTI val Car | AP3D (IoU=0.7) Moderate: 17.05 -> 17.79 | Helped |
| add occupancy learning in 3D space (ablation) | CaDDN-based detector · KITTI val Car | AP3D (IoU=0.7) Moderate: 17.05 -> 18.88 | Helped |
| add occupancy learning in both frustum and 3D space (ablation) | CaDDN-based detector · KITTI val Car | AP3D (IoU=0.7) Moderate: 17.05 -> 19.96 (Easy 21.04 -> 26.87) | Helped |
| occupancy learning (both spaces) vs CaDDN baseline | monocular 3D detector · Waymo val Vehicle LEVEL 1 | 3D mAP IoU=0.5 Overall: 17.54 -> 28.99 | Helped |
| occupancy learning for samples at 50m to infinity | OccupancyM3D vs DID-M3D · Waymo val Vehicle LEVEL 1 [50m, inf) | 3D mAP IoU=0.5: worse than DID-M3D (5.35 -> 3.65) | Hurt |
| occupancy learning vs video-based DfM on KITTI test Car | OccupancyM3D · KITTI test Car | APBEV Moderate: 24.18 vs 22.89 | Helped |
Surprising findings
- Performs worse than DID-M3D for objects beyond 50 m on Waymo at IoU 0.5 due to the voxel-based detection range limit.
Limitations
- Voxel size trades quantization error against computation and GPU memory.
- Voxel-based representation limits detection range (about 2 to 59.6 m on Waymo), hurting far objects.
- Occupancy estimates are poor for heavily occluded objects, which drives most failures.
Experimental setup
Datasets: KITTI Waymo Open Dataset
Benchmarks: KITTI Waymo Open Dataset
Metrics: AP_BEV AP_3D 3D mAP mAPH
Baselines: CaDDN MonoUNI DID-M3D MonoDDE Cube R-CNN NeuROCS MonoJSG DEVIANT DfM
Builds on: CaDDN focal loss DLA34
Closest papers by meaning
- Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data (NeurIPS 2024 · similarity 0.89)
- SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction (CVPR 2024 · similarity 0.88)
- CSV-Occ: Fusing Multi-frame Alignment for Occupancy Prediction with Temporal Cross State Space Model and Central Voting Mechanism (ICML 2025 · similarity 0.88)
- AdaOcc: Adaptive 3D Occupancy Prediction for Embodied Tasks (NeurIPS 2026 · similarity 0.87)
- MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models (CVPR 2024 · similarity 0.87)
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.