Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GenComm, a generation-based communication mechanism where agents send lightweight spatial messages from a deformable message extractor and the ego agent generates collaborator features with a conditional diffusion model, enabling pragmatic heterogeneous collaborative perception.
Problem
Heterogeneous multi-agent collaborative perception suffers from domain gaps across differing sensors and models; existing adaptation and reconstruction methods require intrusive retraining of encoders/fusion modules and incur high cost when new agents join.
Approach
Each ego agent generates features for collaborators locally with a conditional diffusion model conditioned on lightweight spatial messages (extracted by deformable convolution), so only spatial messages are transmitted and the original networks stay frozen; new agents are integrated by fine-tuning only a small message extractor for the receiver, plus a Channel Enhancer refines generated features b
Key results
- On OPV2V-H with AttFuse, GenComm reaches AP70 0.6332 vs 0.5841 for BackAlign on L_P^64-L_S^32.
- Communication volume is reduced to log2 16 vs 22 for MPDA/BackAlign/STAMP.
- Abstract claims an 81% reduction in computational cost and parameter count when incorporating new agents.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GenComm instead of the strongest reported baseline | Heterogeneous collaboration with AttFuse · OPV2V-H L_P^64-L_S^32 | AP50: 0.7873 -> 0.8043 | Helped |
| Use GenComm instead of STAMP | Heterogeneous collaboration with AttFuse · DAIR-V2X L_P^64-L_S^40 | AP50: 0.3913 -> 0.3786 | Hurt |
| Use GenComm instead of STAMP | Heterogeneous collaboration with V2X-ViT · DAIR-V2X L_P^64-L_S^40 | AP50: 0.4935 -> 0.4665 | Hurt |
| Use GenComm instead of STAMP | Dynamic heterogeneous collaboration · OPV2V-H with four agents | AP70: 0.6002 -> 0.6184 | Helped |
| Add collaborator C_E | GenComm · OPV2V-H | AP70: 0.5848 -> 0.5951 (+0.010) | Helped |
| Add collaborator L_S^32 | GenComm · OPV2V-H | AP70: 0.5951 -> 0.6174 (+0.022) | Helped |
| Add collaborator C_R | GenComm · OPV2V-H | AP70: 0.6174 -> 0.6184 (+0.001) | Helped |
| Increase message degradation ratio from 0 to 0.8 | GenComm · V2X-Real with four agents | AP30: 0.714 -> 0.636 | Hurt |
Limitations
- Requires consensus among vendors to collaborate, which may be hindered by commercial competition or malicious attacks.
- Degradation experiments show performance drops with higher message loss ratios.
Experimental setup
Datasets: OPV2V-H DAIR-V2X V2X-Real
Benchmarks: OPV2V-H DAIR-V2X V2X-Real
Metrics: AP30 AP50 AP70 communication volume parameters FLOPs
Baselines: MPDA BackAlign CodeFilling STAMP HEAL
Builds on: HEAL STAMP CodeFilling MPDA PConv
Closest papers by meaning
- An Extensible Framework for Open Heterogeneous Collaborative Perception (ICLR 2024 · similarity 0.91)
- NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception (NeurIPS 2025 · similarity 0.90)
- CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning (AAAI 2025 · similarity 0.88)
- One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception (CVPR 2025 · similarity 0.88)
- Communication-Efficient Collaborative Perception via Information Filling with Codebook (CVPR 2024 · similarity 0.88)
- What Makes Good Collaborative Views? Contrastive Mutual Information Maximization for Multi-Agent Perception (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.