Research map / Detection and segmentation / 3D detection and driving
UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces UAV3D, a simulated (CARLA-AirSim) large-scale benchmark of 1,000 UAV scenes with 3D vehicle boxes, benchmarking single-UAV and collaborative-UAV 3D detection and tracking with PETR, BEVFusion, DETR3D and four collaborative fusion methods.
Problem
Existing UAV benchmarks target 2D perception and single-agent views, limiting 3D and collaborative perception for UAVs over long distances or occlusions.
Approach
A simulated multi-UAV dataset in nuScenes format with five cross-shaped UAVs each carrying five RGB cameras, providing synchronized multi-view images, 3D boxes and tracking labels to benchmark single- and collaborative-UAV 3D detection and tracking.
Key results
- On the UAV3D test set, DETR3D (Res-101, 800x450) reaches the best single-UAV detection with mAP 0.610 and NDS 0.665.
- For collaborative detection on the test set, DiscoNet reaches mAP 0.696 and NDS 0.686, close to the upper-bound raw-image fusion (mAP 0.720, NDS 0.746), versus lower-bound mAP 0.542.
- For collaborative tracking on the test set, DiscoNet achieves AMOTA 0.807 against 0.641 for the lower-bound.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase backbone and image size for DETR3D (ResNet-50, 704×256 to ResNet-101, 800×450) (ablation) | DETR3D · UAV3D validation 3D detection | mAP: gains of approximately 0.18 in mAP | Helped |
| Increase backbone and image size for BEVFusion (ResNet-50, 704×256 to ResNet-101, 800×450) (ablation) | BEVFusion · UAV3D validation 3D detection | mAP: gains of about 0.05 in mAP | Helped |
| Increase backbone and image size for DETR3D (ResNet-50, 704×256 to ResNet-101, 800×450) (ablation) | DETR3D · UAV3D validation single-UAV tracking | AMOTA: gains of approximately 0.18 in AMOTA | Helped |
| Increase backbone and image size for BEVFusion (ResNet-50, 704×256 to ResNet-101, 800×450) (ablation) | BEVFusion · UAV3D validation single-UAV tracking | AMOTA: about 0.04 in AMOTA | Helped |
| Use ResNet-101 and 800×450 input instead of ResNet-50 and 704×256 (ablation) | DETR3D · UAV3D V1.0-mini validation detection | mAP: 0.023 to 0.051 | Helped |
| Use ResNet-101 and 800×450 input instead of ResNet-50 and 704×256 (ablation) | PETR · UAV3D V1.0-mini validation detection | mAP: 0.003 to 0.005 | No clear effect |
| Use ResNet-101 and 800×450 input instead of ResNet-50 and 704×256 (ablation) | BEVFusion · UAV3D V1.0-mini validation detection | mAP: 0.113 to 0.112 | No clear effect |
| Add collaboration using raw images from all five UAVs instead of no collaboration | Collaborative-UAV 3D detection · UAV3D validation | mAP: 0.18 | Helped |
| Add DiscoNet intermediate-feature collaboration instead of no collaboration | Collaborative-UAV 3D detection · UAV3D validation | mAP: 0.544 -> 0.700 | Helped |
| Add When2Com attention-based collaboration instead of no collaboration | Collaborative-UAV 3D detection · UAV3D validation | mAP: 0.544 -> 0.550 | Helped |
| Add Who2Com collaboration instead of no collaboration | Collaborative-UAV 3D detection · UAV3D validation | mAP: 0.544 -> 0.546 | Helped |
| Add raw-image collaboration instead of no collaboration | Collaborative-UAV tracking · UAV3D validation | AMOTA: about 0.17 in AMOTA | Helped |
Surprising findings
- PETR with ResNet-101 performed worse than with ResNet-50, which the authors state they chose not to report in the paper.
- When2Com and Who2Com give only marginal gains over the single-UAV lower-bound in collaborative detection.
Limitations
- Simulated data only; sim-to-real domain gap is not evaluated.
- Only clear-weather, daytime conditions in the current version.
- Only a subset of recent 3D detection/tracking models (PETR, BEVFusion, DETR3D) are benchmarked.
Experimental setup
Datasets: UAV3D
Benchmarks: UAV3D
Metrics: mAP NDS mATE mASE mAOE AMOTA AMOTP MOTA MOTP TID
Baselines: PETR BEVFusion DETR3D When2Com Who2Com V2VNet DiscoNet
Builds on: nuScenes CARLA AirSim V2X-Sim BEVFusion
Closest papers by meaning
- AGC-Drive: A Large-Scale Dataset for Real-World Aerial-Ground Collaboration in Driving Scenarios (NeurIPS 2025 · similarity 0.87)
- Multiview Aerial Visual RECognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? (CVPR 2024 · similarity 0.85)
- MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking (NeurIPS 2025 · similarity 0.85)
- MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking (CVPR 2025 · similarity 0.85)
- WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife (NeurIPS 2026 · similarity 0.85)
- Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.