Research map / Multi-agent systems and games / Human-AI and goal-conditioned
An Extensible Framework for Open Heterogeneous Collaborative Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEAL, an extensible heterogeneous collaborative perception framework that builds a unified BEV feature space with a multiscale foreground-aware Pyramid Fusion and integrates new agent types via backward alignment (training only the new encoder), plus a new OPV2V-H dataset.
Problem
Collaborative perception where new heterogeneous agent types (unseen sensor modalities or detection models) continually join an existing collaboration, with a domain gap, while keeping performance high and integration cost low.
Approach
Establish a unified BEV feature space with a pretrained Pyramid Fusion module and detection head, then train only the new agent type's encoder on single agents so its features align to the frozen unified space (backward alignment), avoiding collective retraining.
Key results
- On OPV2V-H with 3 added agent types, HEAL reaches AP70 0.813 vs HM-ViT 0.755 while using 8.3% of HM-ViT's parameters for the integration (91.5% reduction).
- On DAIR-V2X with 2 agents, HEAL achieves AP50 0.770 for L(40)P+C(288)E versus HM-ViT 0.638 with lower accumulated parameters.
- Pyramid Fusion on homogeneous OPV2V LiDAR reaches AP70 0.926 vs HM-ViT 0.873.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HEAL rather than HM-ViT when adding C(336)_R | HEAL · OPV2V-H, third new agent integration | AP70: 0.755 -> 0.813 | Helped |
| Use HEAL rather than HM-ViT when adding C(336)_R | HEAL · OPV2V-H, third new agent integration | Model #Params (M): 83.34 -> 7.15 | Helped |
| Use HEAL instead of HM-ViT | HEAL · DAIR-V2X, L(40)_P+L(40)_S | AP50: 0.538 -> 0.770 | Helped |
| Add the multiscale pyramid (ablation) | HEAL without backward alignment · OPV2V-H heterogeneous collaboration | AP70: 0.578 -> 0.628 | Helped |
| Add foreground supervision (ablation) | HEAL without backward alignment · OPV2V-H heterogeneous collaboration | AP70: 0.578 -> 0.633 | Helped |
| Add foreground supervision (ablation) | HEAL with backward alignment and multiscale pyramid · OPV2V-H heterogeneous collaboration | AP70: 0.781 -> 0.813 | Helped |
| Add the multiscale pyramid (ablation) | HEAL with backward alignment and foreground supervision · OPV2V-H heterogeneous collaboration | AP70: 0.782 -> 0.813 | Helped |
| Add backward alignment (ablation) | HEAL with multiscale pyramid and foreground supervision · OPV2V-H heterogeneous collaboration | AP70: 0.662 -> 0.813 | Helped |
| Use multiscale pyramid and foreground supervision together rather than either alone (ablation) | HEAL without backward alignment · OPV2V-H heterogeneous collaboration | AP70: 0.628 -> 0.662; 0.633 -> 0.662 | Helped |
Surprising findings
- Late fusion's accumulated parameters and training cost are lower than intermediate methods, yet its AP70 is much lower than HEAL.
- Even with 32-fold feature compression, HEAL retains high performance and surpasses baselines.
Limitations
- Requires BEV features, so models without BEV output (e.g., keypoint-based LiDAR detectors) need extra adaptation.
- Evaluated mainly on simulated OPV2V-H with DAIR-V2X as the only real-world dataset with at most 2 agents.
Experimental setup
Datasets: OPV2V OPV2V-H DAIR-V2X
Benchmarks: OPV2V-H DAIR-V2X OPV2V
Metrics: AP50 AP70 AP30 training parameters FLOPs peak memory training throughput
Baselines: Late Fusion No Fusion F-Cooper DiscoNet AttFusion V2X-ViT CoBEVT HM-ViT
Builds on: PointPillars SECOND Lift-Splat OPV2V HM-ViT
Closest papers by meaning
- Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism (NeurIPS 2025 · similarity 0.91)
- NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception (NeurIPS 2025 · similarity 0.91)
- CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning (AAAI 2025 · similarity 0.91)
- One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception (CVPR 2025 · similarity 0.91)
- CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridization (CVPR 2025 · similarity 0.89)
- What Makes Good Collaborative Views? Contrastive Mutual Information Maximization for Multi-Agent Perception (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.