Research map / 3D vision / 3D generation
MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MonoDiff, a diffusion-based monocular 3D object detector and pose estimator that uses a GMM-initialized reverse diffusion process and 2D reprojection supervision, using no extra modalities or depth priors.
Problem
Monocular 3D object detection and orientation estimation from a single image is highly uncertain; existing top methods rely on extra modalities (stereo, LiDAR) or annotations, and standard Gaussian-initialized diffusion does not fit the varying box-dimension uncertainty.
Approach
Model the 3D box parameters (dimensions and rotation) with a diffusion process whose forward noise and reverse initialization come from a Gaussian mixture fitted to ground-truth boxes, plus a 2D reprojection loss weighted by SNR-based timestep weights.
Key results
- On KITTI test Car, MonoDiff reaches 93.72 moderate AOS / 21.02 AP3D vs MonoNeRD 86.13 / 17.13.
- On Waymo val LEVEL 1, MonoDiff gets 32.28 overall 3D mAP vs MonoNeRD 31.18.
- Ablation on KITTI val shows GMM initialization adds about 3% orientation and about 2% detection gains over standard diffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MonoDiff instead of MonoNeRD | KITTI Car detection · KITTI test, Car Moderate | AP3D: 17.13 -> 21.02 | Helped |
| Use MonoDiff instead of MonoNeRD | Waymo object detection · Waymo val, Level 1 Overall | 3D mAP: 31.18 -> 32.28 | Helped |
| Use iterative diffusion rather than single-step inference (ablation) | MonoDiff · KITTI val | AP3D Moderate: 18.25 -> 22.02 | Helped |
| Use the generative diffusion process rather than an iterated single-step model (ablation) | MonoDiff · KITTI val | AP3D Moderate: 18.47 -> 22.02 | Helped |
| Add GMM initialization (ablation) | Standard diffusion · KITTI val | AP3D Moderate: 19.99 -> 21.44 | Helped |
| Add 2D projection supervision (ablation) | GMM-initialized MonoDiff · KITTI val | AP3D Moderate: 21.44 -> 21.62 | Helped |
| Add feature encoder conditioning (ablation) | GMM-initialized MonoDiff with 2D projection · KITTI val | AP3D Moderate: 21.62 -> 21.82 | Helped |
| Add timestep weighting (ablation) | MonoDiff · KITTI val | AP3D Moderate: 21.82 -> 22.02 | Helped |
| Use 5 rather than 3 GMM components (ablation) | MonoDiff · KITTI val | AP3D Moderate: 20.62 -> 21.44 | Helped |
| Use 8 rather than 5 GMM components (ablation) | MonoDiff · KITTI val | AP3D Moderate: 21.44 -> 21.53 | Helped |
| Replace CenterNet with YOLOv7 (ablation) | MonoDiff · KITTI val | AP3D Moderate: 22.02 -> 22.12 | Mixed |
| Replace ResNet-34 with ResNet-50 (ablation) | MonoDiff · KITTI val | AP3D Moderate: 22.02 -> 22.16 | Helped |
Surprising findings
- Lighter 2D detectors and backbones (e.g., MobileNet, RetinaNet) perform comparably, and in some Easy-class cases outperform heavier ones.
Limitations
- Uses a fixed 2D detector, so end-to-end training is not possible.
- Speed depends on the number of reverse diffusion steps, trading off against accuracy.
Experimental setup
Datasets: KITTI Waymo Open Dataset
Benchmarks: KITTI 3D detection Waymo Open Dataset
Metrics: AP3D (R40) AOS 3D mAP 3D mAPH
Baselines: MonoNeRD LPCG CMKD DID-M3D MonoJSG CaDDN MonoRCNN D4LCN DDMP-3D
Builds on: DDPM DDIM DiffusionDet Min-SNR weighting
Closest papers by meaning
- MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors (CVPR 2025 · similarity 0.91)
- MonoCD: Monocular 3D Object Detection with Complementary Depths (CVPR 2024 · similarity 0.89)
- MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection (CVPR 2025 · similarity 0.89)
- DiffuBox: Refining 3D Object Detection with Point Diffusion (NeurIPS 2024 · similarity 0.89)
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders (NeurIPS 2024 · similarity 0.89)
- Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.