Research map / Detection and segmentation / 3D detection and driving
Towards Generalizable Multi-Camera 3D Object Detection via Perspective Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PD-BEV, a model-agnostic perspective-debiasing framework that renders multi-view semantic maps from BEV features via implicit foreground volumes to improve domain generalization and unsupervised adaptation of multi-camera 3D detection, with no added inference cost.
Problem
Multi-camera 3D object detection (MC3D-Det) with BEV methods degrades severely under domain shifts (viewpoints, camera parameters, environments), and labeled diverse target data is scarce.
Approach
Lift BEV features into an implicit foreground volume, render camera-plane heatmaps from randomly perturbed virtual cameras, and supervise them with source 3D boxes (and with a pre-trained 2D detector's pseudo-heatmaps on the target domain), so the BEV detector learns perspective- and context-invariant features; the extra heads are dropped at inference.
Key results
- On Nus→Lyft DG, PD-BEV raises target-domain mAP from 0.114 (BEVDepth) to 0.304 and NDS* from 0.296 to 0.458 for car.
- Under UDA, PD-BEV+ reaches target mAP 0.316 and NDS* 0.476 on Nus→Lyft, versus 0.598 mAP oracle-level target result of 0.598 for the Lyft-trained oracle on the source side.
- Plugging the framework into BEVDet on Nus→Lyft improves target mAP from 0.104 to 0.296 and NDS* from 0.275 to 0.446.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add perspective debiasing (PD-BEV) | BEVDepth · nuScenes→Lyft (DG) | mAP: 0.114 -> 0.304 | Helped |
| Add perspective debiasing (PD-BEV) | BEVDepth · Lyft→nuScenes (DG) | mAP: 0.098 -> 0.263 | Helped |
| Add perspective debiasing (PD-BEV) | BEVDepth · DeepAccident→nuScenes (DG) | mAP: 0.087 -> 0.187 | Helped |
| Add perspective debiasing (PD-BEV) | BEVDepth · DeepAccident→Lyft (DG) | mAP: 0.045 -> 0.151 | Helped |
| Add target-domain debiasing with a 2D detector (ablation) | PD-BEV · nuScenes→Lyft (UDA) | mAP: 0.304 -> 0.316 | Helped |
| Add target-domain debiasing with a 2D detector (ablation) | PD-BEV · DeepAccident→Lyft (UDA) | mAP: 0.151 -> 0.171 | Helped |
| Add 2D detector pre-training (DPT) (ablation) | PD-BEV · nuScenes→Lyft | mAP: 0.279 -> 0.290 | Helped |
| Add source-domain debiasing (SDB) (ablation) | PD-BEV · nuScenes→Lyft | mAP: 0.279 -> 0.300 | Helped |
| Use box-based rather than LiDAR depth supervision (ablation) | PD-BEV+ · DeepAccident→Lyft | mAP: 0.148 -> 0.171 | Helped |
| Use box-based rather than LiDAR depth supervision (ablation) | PD-BEV+ · nuScenes→Lyft | mAP: 0.312 -> 0.316 | Helped |
| Use LiDAR depth supervision rather than no depth supervision (ablation) | PD-BEV+ · DeepAccident→Lyft | mAP: 0.162 -> 0.148 | Hurt |
Surprising findings
- Training with no depth supervision outperforms LiDAR depth supervision under sim-to-real (DeepAccident→Lyft), per the authors' depth-supervision comparison.
- Coral and AD improve performance in virtual-to-real transfer but degrade it in real-to-real transfer, since they may disrupt semantics when style shifts are small.
Limitations
- Perspective-bias derivation is simplified (horizontal-plane camera assumption) and the authors note the full analysis is too complicated.
- Gains on the source domain are small and UDA can degrade source performance for some baselines.
- Evaluation restricted to a car-only NDS* for cross-dataset settings and to BEVDepth-based instantiation for main results.
Experimental setup
Datasets: nuScenes Lyft DeepAccident
Benchmarks: nuScenes Lyft DeepAccident
Metrics: mAP mATE mASE mAOE NDS*
Baselines: BEVDepth DG-BEV Pseudo Label Coral AD BEVDet BEVFormer FB-OCC
Builds on: BEVDepth DG-BEV CenterPoint Coral
Closest papers by meaning
- Unified Domain Generalization and Adaptation for Multi-View 3D Object Detection (NeurIPS 2024 · similarity 0.91)
- CaMuViD: Calibration-Free Multi-View Detection (CVPR 2025 · similarity 0.89)
- GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection (AAAI 2025 · similarity 0.89)
- UniMODE: Unified Monocular 3D Object Detection (CVPR 2024 · similarity 0.88)
- Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning (ICML 2025 · similarity 0.88)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.