Research map / Detection and segmentation / 3D detection and driving
Instance-Aware Multi-Camera 3D Object Detection with Structural Priors Mining and Self-Boosting Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IA-BEV, which injects 2D instance awareness into depth estimation of a BEV multi-camera detector via category-specific monocular depth decoders with instance-level losses (SPM) and iterative sparse temporal stereo matching focused on uncertain objects (SBL).
Problem
Camera-based BEV 3D detection relies on depth estimation, but existing methods predict depth for every pixel equally without prioritizing foreground objects, and stereo cost volumes over dense pixels are memory-expensive.
Approach
Parse foreground instances with a 2D segmentor, use category-grouped lightweight depth decoders conditioned on box size with instance-aware absolute and relative depth losses (SPM), and iteratively refine sparse stereo matching on high-uncertainty object pixels with denser hypotheses (SBL).
Key results
- On nuScenes test with ConvNeXt-B at 640x1600, IA-BEV reaches 0.545 mAP and 0.630 NDS, the best among two-frame methods reported.
- On nuScenes val with ResNet-50 at 256x704, IA-BEV reaches 0.400 mAP and 0.516 NDS, versus 0.351 mAP and 0.475 NDS for BEVDepth.
- Ablation: SPM alone gives +1.8% NDS and SBL alone +2.1% NDS over the baseline; combined they raise NDS from 42.5% to 46.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IA-BEV depth estimation components | BEVDepth-R50 · nuScenes val, ResNet-50, 256x704 | mAP: 0.387 -> 0.400 | Helped |
| add IA-BEV depth estimation components | BEVDepth-ConvNeXt-B · nuScenes val, ConvNeXt-B, 512x1408 | NDS: 0.579 -> 0.581 | Helped |
| add IA-BEV depth estimation components | BEVDepth-ConvNeXt-B · nuScenes test, ConvNeXt-B, 640x1600 | mAP: 0.540 -> 0.545 | Helped |
| add SPM (ablation) | BEVDepth · nuScenes validation ablation | NDS: 0.425 -> 0.443 | Helped |
| add SBL (ablation) | BEVDepth · nuScenes validation ablation | NDS: 0.425 -> 0.446 | Helped |
| combine SPM and SBL (ablation) | BEVDepth · nuScenes validation ablation | NDS: 0.425 -> 0.461 | Helped |
| add parallel decoders without semantic grouping (ablation) | BEVDepth · nuScenes SPM ablation | mAP: 0.330 -> 0.329 | Hurt |
| use category-specific decoders instead of parallel decoders without grouping (ablation) | SPM · nuScenes SPM ablation | mAP: 0.330 -> 0.336 | Helped |
| add instance-level absolute depth supervision (ablation) | category-specific decoders · nuScenes SPM ablation | mAP: 0.336 -> 0.340 | Helped |
| add inner-instance relative depth supervision (ablation) | category-specific decoders with absolute depth supervision · nuScenes SPM ablation | mAP: 0.340 -> 0.345 | Helped |
| add one SBL iteration (ablation) | BEVDepth · nuScenes SBL ablation | mAP: 0.330 -> 0.346 | Helped |
| add a second SBL iteration (ablation) | SBL with one iteration · nuScenes SBL ablation | mAP: 0.346 -> 0.354 | Helped |
Surprising findings
- Simply increasing the number of parallel depth decoders without category grouping gives no benefit over the baseline.
Limitations
- Gains from iterating beyond 2 rounds are not significant, attributed to current resolution and feature quality limits.
- Depends on off-the-shelf 2D instance segmentor to parse foreground objects.
- Ground-truth depths from LiDAR contain outliers due to imperfect calibration, requiring heuristic voting.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes val nuScenes test
Metrics: mAP NDS mATE mASE mAOE mAVE mAAE SILog AbsRel SqRel
Baselines: BEVDepth BEVStereo BEVDet4D PETR AeDet FB-BEV SA-BEV OA-BEV MV2D SOLOFusion BEVFormer PETRv2
Builds on: BEVDepth Lift-Splat-Shoot MonoDETR BEVStereo
Closest papers by meaning
- UniMODE: Unified Monocular 3D Object Detection (CVPR 2024 · similarity 0.88)
- Towards Generalizable Multi-Camera 3D Object Detection via Perspective Rendering (AAAI 2025 · similarity 0.88)
- GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection (AAAI 2025 · similarity 0.88)
- IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection (CVPR 2024 · similarity 0.87)
- Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting (CVPR 2025 · similarity 0.87)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.