Research map / Vision-language and multimodal / Multimodal LLMs

AirCopBench: A Benchmark for Multi-drone Collaborative Embodied Perception and Reasoning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AirCopBench, a 14.6k-question VQA benchmark from simulated and real multi-drone imagery with degraded perception, evaluating 40 MLLMs on scene, object, perception assessment and collaborative decision tasks, plus SFT and sim-to-real experiments.

Problem

Existing multi-image benchmarks focus on basic perception with high-quality single-agent images, and collaborative perception benchmarks for aerial agents lack semantic VQA and degraded-perception conditions, so MLLMs' embodied multi-drone collaborative perception is unevaluated.

Approach

A benchmark of 14 VQA task types across four dimensions built from simulator, real-world, and noise/masking-derived multi-view UAV images with event-level collaboration labels, generated by model-, rule-, and human-based methods with blind filtering and human refinement.

Key results

• Best zero-shot model Ovis2-16B reaches 59.17% accuracy on AirCopBench, while human baseline averages 78.25%.
• Fine-tuning Qwen-2.5-VL-7B on the curated instruction data raises accuracy from about 48% to 74.30% (+26.97).
• Sim-to-real: AirCop-7B, fine-tuned on simulator data, reaches 67.41% on real-world UAV imagery versus 47.77% for Qwen2.5-VL-7B.

Surprising findings

• Open-source models such as Ovis2-16B and Kimi-VL-A3B-Thinking match or slightly surpass top proprietary models.
• Current MLLMs perform well on scene description but poorly on usability assessment and when-to-collaborate questions.

Limitations

• Real-world data is limited to representative images from one dataset (MDMT) with 2 UAVs, while most data is simulated.
• Some questions are generated by GPT-4o and Qwen-VL-Max, so model-generated content may carry biases despite quality control.

Experimental setup

Datasets: AirCopBench Coperception-UAV AeroCollab3D EmbodiedCity MDMT

Benchmarks: AirCopBench

Metrics: accuracy

Baselines: GPT-4o Gemini-2.5-Pro Claude-Sonnet-4 Qwen-Max-VL Qwen2.5-VL-7B Qwen2.5-VL-72B InternVL3-78B Ovis2-16B LLaVA-NeXT-13B Phi-4-multimodal-instruct

Builds on: VLMEvalKit LLaMA-Factory

Closest papers by meaning

• AGC-Drive: A Large-Scale Dataset for Real-World Aerial-Ground Collaboration in Driving Scenarios (NeurIPS 2025 · similarity 0.88)
• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.87)
• EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents (NeurIPS 2026 · similarity 0.87)
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.87)
• PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (ICLR 2025 · similarity 0.87)
• PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments (NeurIPS 2025 · similarity 0.87)