Research map / Detection and segmentation / 3D detection and driving
Unified Domain Generalization and Adaptation for Multi-View 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UDGA, combining a multi-view overlap depth constraint for domain generalization in camera-based multi-view 3D detection with a label-efficient adapter-based domain adaptation (1%/5% target labels).
Problem
Camera-based multi-view 3D object detection degrades under geometric (extrinsic/installation) shifts between source and target datasets, and adapting to new domains is costly in labels and compute.
Approach
Constrains depth predictions to agree across overlapping regions of adjacent cameras (and photometric consistency) to learn perspective-invariant depth for domain generalization, then fine-tunes only lightweight bottleneck adapters parallel to frozen pre-trained blocks for label-efficient adaptation with few target labels.
Key results
- Lyft→nuScenes domain generalization NDS* 0.421 vs DG-BEV 0.374 and direct transfer 0.213.
- With 1% target labels UDGA reaches NDS* 0.526 on Lyft→nuScenes, and 0.578 on nuScenes→Lyft, versus Full FT 0.476 and 0.531.
- Adapter-based UDGA uses 8.8M trainable parameters (less than 20% of the total) and matches or exceeds Full FT at 100% target data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add UDGA's overlap-depth constraint (ablation) | BEVDepth domain generalization · Lyft → nuScenes | NDS: 0.213 -> 0.403 | Helped |
| add photometric loss to overlap-depth loss (ablation) | BEVDepth domain generalization · Lyft → nuScenes | NDS: 0.403 -> 0.407 | Helped |
| add extrinsic augmentation to overlap-depth and photometric losses (ablation) | BEVDepth domain generalization · Lyft → nuScenes | NDS: 0.407 -> 0.421 | Helped |
| add photometric loss to overlap-depth loss (ablation) | BEVDepth domain generalization · nuScenes → Lyft | NDS: 0.485 -> 0.487 | Helped |
| add extrinsic augmentation to overlap-depth and photometric losses (ablation) | BEVDepth domain generalization · nuScenes → Lyft | NDS: 0.487 -> 0.488 | Helped |
| use overlap-depth and photometric losses in both source pre-training and target fine-tuning (ablation) | UDGA · Lyft → nuScenes, 10% target labels | NDS: 0.516 -> 0.638 | Helped |
| add overlap-depth loss during target fine-tuning (ablation) | UDGA · nuScenes → Lyft, 10% target labels | NDS: 0.590 -> 0.632 | Helped |
| add photometric loss during target fine-tuning (ablation) | UDGA · nuScenes → Lyft, 10% target labels | NDS: 0.632 -> 0.573 | Hurt |
| add UDGA domain generalization | BEVDepth · Lyft → nuScenes | NDS: 0.374 -> 0.421 | Helped |
| adapt with UDGA using 1% target labels instead of full fine-tuning | BEVDepth · Lyft → nuScenes | NDS: 0.476 -> 0.526 | Helped |
| adapt with UDGA using 5% target labels instead of full fine-tuning | BEVDepth · nuScenes → Lyft | NDS: 0.594 -> 0.613 | Helped |
| attach adapters to the view transform, BEV encoder, and detection head rather than the detection head alone (ablation) | UDGA · Lyft → nuScenes, 5% target labels | NDS: 0.333 -> 0.525 | Helped |
Surprising findings
- Adapter-based fine-tuning without the proposed depth constraint (Adapter-B, Adapter-S) underperforms Full FT and sometimes Scratch on Lyft→nuScenes at 100% target data.
- Attaching adapters only at the detection head degrades Lyft→nuScenes performance.
Limitations
- Performance remains below LiDAR-based 3D detection models.
- The overlap depth constraint requires overlapping regions between adjacent camera views.
- Fully domain-agnostic adaptation without any target labels remains unsolved.
Experimental setup
Datasets: nuScenes Lyft Waymo CARLA
Benchmarks: nuScenes Lyft Waymo
Metrics: NDS* mAP mATE mASE mAOE Closed Gap
Baselines: Direct Transfer Oracle CAM-Convs Single-DGOD DG-BEV PD-BEV Full FT SSF Adapter-B Adapter-S
Builds on: BEVDepth DG-BEV Adapter SSIM photometric loss
Closest papers by meaning
- Towards Generalizable Multi-Camera 3D Object Detection via Perspective Rendering (AAAI 2025 · similarity 0.91)
- Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment (CVPR 2024 · similarity 0.90)
- Geometry-Guided Domain Generalization for Monocular 3D Object Detection (AAAI 2024 · similarity 0.89)
- Pseudo Label Refinery for Unsupervised Domain Adaptation on Cross-dataset 3D Object Detection (CVPR 2024 · similarity 0.88)
- Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning (ICML 2025 · similarity 0.88)
- Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.