Research map / Detection and segmentation / 3D detection and driving
MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
MonoMAE masks non-occluded object queries in feature space with depth-aware ratios, then reconstructs them with a lightweight completion network to learn occlusion-tolerant monocular 3D detection.
Problem
Monocular 3D object detection degrades substantially under object occlusion, which harms predictions of depth, dimensions, and orientation.
Approach
Treat occlusion as natural masking: adaptively mask non-occluded object queries in feature space with a depth-dependent ratio (closer objects masked more), and train a lightweight completion network to reconstruct them, so occluded-looking query pairs supervise training with no masking at inference.
Key results
- On the KITTI 3D test set, MonoMAE reports AP_3D (R40, IoU 0.7) of 25.60 / 18.84 / 16.78 (Easy/Moderate/Hard) without extra data, above MonoCD (25.53 / 16.59 / 14.53) and MonoDETR (25.00 / 16.47 / 13.58).
- On KITTI val, the full model reaches AP_3D 30.29 / 20.90 / 17.61 versus 24.85 / 16.21 / 14.74 for the NOQG-only baseline in the ablation.
- Depth-aware query masking improves over random query masking (AP_3D Moderate 20.90 vs 18.47) and over image-level masking (15.03), and inference time is 38 ms versus 43 ms for MonoDETR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MonoMAE's depth-aware masking and query completion | MonoMAE · KITTI 3D test, Moderate | AP3D: 16.47 -> 18.84 | Helped |
| Add the Completion Network to NOQG + DAM (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 15.71 -> 20.90 | Helped |
| Add the Completion Network to the NOQG baseline (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 16.21 -> 18.76 | Helped |
| Add NOQG to DAM + CN (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 19.35 -> 20.90 | Helped |
| Mask query features instead of input images (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 15.03 -> 20.90 | Helped |
| Add depth-aware masking to query masking (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 18.47 -> 20.90 | Helped |
| Add L_com to L_occ (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 19.61 -> 20.90 | Helped |
| Add L_occ to L_com (ablation) | MonoMAE · KITTI 3D val, Moderate | AP3D: 19.15 -> 20.90 | Helped |
| Remove the Completion Network at inference (ablation) | MonoMAE · KITTI val | inference time: 38 -> 36 | Helped |
| Transfer the KITTI-trained model to nuScenes without additional training | MonoMAE · nuScenes frontal val, all depths | mean absolute depth error: 1.40 vs 1.26 | Mixed |
Surprising findings
- Masking naturally occluded queries to train the completion network hurts performance, so masking is applied only to non-occluded queries.
- Image-level masking performs worse than random feature-space masking, suggesting reconstruction in image space is harder with a lightweight network.
Limitations
- Masked non-occluded queries may differ in pattern from naturally occluded query features, creating a train-test gap.
- Reported completion network size (2.22G parameters) appears inconsistent with a lightweight design and FLOPs figure as stated.
Experimental setup
Datasets: KITTI 3D nuScenes
Benchmarks: KITTI 3D nuScenes
Metrics: AP_3D R40 (IoU 0.7) AP_BEV R40 (IoU 0.7) mean absolute depth error inference time
Baselines: MonoDETR GUPNet MonoCD MonoUNI DEVIANT MonoDTR MonoNeRD DID-M3D MonoFlex MonoRCNN
Builds on: Masked Autoencoders MonoDETR GUPNet bipartite matching
Closest papers by meaning
- MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors (CVPR 2025 · similarity 0.90)
- MonoCD: Monocular 3D Object Detection with Complementary Depths (CVPR 2024 · similarity 0.90)
- FD3D: Exploiting Foreground Depth Map for Feature-Supervised Monocular 3D Object Detection (AAAI 2024 · similarity 0.89)
- MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models (CVPR 2024 · similarity 0.89)
- RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection (NeurIPS 2026 · similarity 0.88)
- Weakly Supervised Monocular 3D Detection with a Single-View Image (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.