Research map / 3D vision / 3D generation
MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a scene-aware distribution over plausible 3D car boxes from a road image and renders realistic ControlNet-edited cars at those locations to augment KITTI/NuScenes training for monocular 3D detection.
Problem
Monocular 3D detectors are limited by scarce and low-diversity real data, and heuristic placement of synthetic objects in outdoor road scenes yields unrealistic augmentations that give marginal gains.
Approach
Learn where and how cars should be placed (3D box distribution) from inpainted KITTI scenes using geometry-aware neighbor-interpolation augmentation and Gaussian reparameterized box predictions, then render cars with edge-conditioned ControlNet (LoRA-finetuned) and composite them into the scene.
Key results
- On KITTI, MonoDLE 3D AP@0.7 easy improves from 17.45 to 22.49 with MonoPlace3D augmentation, outperforming the best baseline RBP (20.50).
- On KITTI, GUPNet 3D AP@0.7 moderate improves from 16.46 to 17.28 and hard from 13.27 to 14.71, while all baselines fail to improve GUPNet overall.
- On NuScenes with FCOS3D, mAP improves from 0.3430 to 0.3704 and NDS from 0.415 to 0.440.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learned scene-aware placement and realistic car rendering for augmentation | MonoDLE · KITTI val, Easy | 3D AP @ IoU=0.7: 17.45 -> 22.49 | Helped |
| Use learned scene-aware placement and realistic car rendering for augmentation | GUPNet · KITTI val, Moderate | 3D AP @ IoU=0.7: 16.46 -> 17.28 | Helped |
| Use MonoPlace3D augmentation | FCOS3D · NuScenes val | mAP: 0.3430 -> 0.3704 | Helped |
| Use MonoPlace3D augmentation | FCOS3D · NuScenes val | NDS: 0.415 -> 0.440 | Helped |
| Add realistic text-to-image car rendering instead of ShapeNet rendering (ablation) | MonoDLE with learned placement · KITTI val, Easy | 3D AP @ IoU=0.7: 20.91 -> 22.49 | Helped |
| Add realistic text-to-image car rendering instead of Lift3D rendering (ablation) | MonoDLE with learned placement · KITTI val, Easy | 3D AP @ IoU=0.7: 21.35 -> 22.49 | Helped |
| Add rendered shadows (ablation) | MonoDLE with MonoPlace3D rendering · KITTI val, Easy | 3D AP @ IoU=0.7: 21.45 -> 22.49 | Helped |
| Add rendered shadows (ablation) | MonoDLE with MonoPlace3D rendering · KITTI val, Moderate | 3D AP @ IoU=0.7: 14.21 -> 15.44 | Helped |
| Add geometric augmentation to placement training (ablation) | SA-PlaceNet · KITTI road-scene placement | plausible placement coverage | Helped |
| Add variational modeling of 3D boxes (ablation) | SA-PlaceNet · KITTI road-scene placement | placement diversity | Helped |
Surprising findings
- Most baseline augmentations (Lift3D, CARLA, Geo-CP) give marginal or negative gains on GUPNet, despite similar augmentation counts.
- Using only about 40-50% of real KITTI training data with the augmentations matches or exceeds the model trained on full data without augmentation.
Limitations
- Depends on an off-the-shelf inpainting method to prepare placement training data.
- Does not model appearance factors such as scene lighting.
- Evaluated mainly on cars; other categories are handled with a simpler copy-paste variant.
Experimental setup
Datasets: KITTI NuScenes ShapeNet KITTI car subset (for LoRA)
Benchmarks: KITTI NuScenes
Metrics: 3D AP@IOU=0.7 3D AP@IOU=0.5 mAP NDS FID
Baselines: Geo-CP CARLA Lift3D Rule Based Placement ShapeNet w/o 3D Augmentation
Builds on: MonoDTR ControlNet Geo-CP Lift3D MonoDLE
Closest papers by meaning
- MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models (CVPR 2024 · similarity 0.89)
- MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors (CVPR 2025 · similarity 0.89)
- LabelAny3D: Label Any Object 3D in the Wild (NeurIPS 2025 · similarity 0.89)
- RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection (NeurIPS 2026 · similarity 0.88)
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders (NeurIPS 2024 · similarity 0.88)
- MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.