Research map / Graph learning / Knowledge and temporal graphs
Asynchronous Collaborative Graph Representation for Frames and Events
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ACGR, a unified graph representation that fuses asynchronous frames and events via offset-based alignment and adversarial domain adaptation, enabling object detection and depth estimation up to 200 Hz.
Problem
Frame-event fusion methods convert events into image-like tensors synchronized with frames, limiting inference to the frame rate, use costly two-branch backbones, and fail to exploit the sparse spatiotemporal structure of events.
Approach
Build sparse unimodal graphs for frames and events, connect them with an offset-estimated cross-modal edge set into one unified graph processed by spline GCNs, and align frame and event feature spaces with adversarial domain adaptation (discarded at inference).
Key results
- On DSEC-Detection, ACGR with YOLOX reaches 0.313 mAP vs 0.266 for the frame-only YOLOX baseline, with 13.2 ms runtime.
- Applying ACGR to DAGr and SODFormer improves mAP by 2.6% and 3.6% while cutting inference time to 0.38x and 0.49x of their baselines.
- Achieves real-time asynchronous inference up to 200 Hz (4.96 ms per input) and best Abs.Rel. on MVSEC outdoor day1 and night1 with 3.3M parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add frame and event graph representation using ACGR | SODFormer · DSEC-Detection | mAP: 0.281 -> 0.317 | Helped |
| add frame and event graph representation using ACGR | YOLOX · DSEC-Detection | mAP: 0.266 -> 0.313 | Helped |
| add ACGR to Litemono | Litemono · MVSEC outdoor day1 | Abs.Rel.: 0.305 -> 0.256 | Helped |
| add event streams (ablation) | YOLOX frame-only baseline · DSEC-Detection | mAP: 0.266 -> 0.284 | Helped |
| add ACAM for asynchronous frame-event alignment (ablation) | YOLOX with event streams · DSEC-Detection | mAP: 0.284 -> 0.291 | Helped |
| add domain alignment loss Ld (ablation) | YOLOX with events and ACAM · DSEC-Detection | mAP: 0.291 -> 0.313 | Helped |
| replace ACGR with frame and event image representations (ablation) | SODFormer · DSEC-Detection | mAP: 0.317 -> 0.281 | Hurt |
| replace ACGR with frame and voxel grid representations (ablation) | SODFormer · DSEC-Detection | mAP: 0.317 -> 0.279 | Hurt |
| replace ACGR with frame and voxel graph representations (ablation) | SODFormer · DSEC-Detection | mAP: 0.317 -> 0.284 | Hurt |
| vary node-filtering threshold θ (ablation) | ACGR with DAGr · DSEC-Detection | mAP: 0.167 (θ=0); 0.282 (θ=1); 0.284 (θ=2); 0.261 (θ=4); 0.259 (θ=8) | Mixed |
| use asynchronous inference with events at 50ms latency (ablation) | ACGR with DAGr · DSEC-Detection | mAP: 0.284 -> 0.283 | No clear effect |
| increase event-node filtering threshold from θ=0 to θ=2 (ablation) | ACGR with DAGr · DSEC-Detection | Runtime (ms): 44.5 -> 9.1 | Helped |
Surprising findings
- Asynchronous inference performance of SODFormer and YOLOX drops as latency grows, whereas the graph-based ACGR stays stable (0.284 to 0.283 mAP across 0-50 ms).
Limitations
- Validated only on object detection and monocular depth estimation.
- Depth estimation on night sequences is sub-optimal in RMS and RMSlog versus event-only EReFormer, attributed to the daytime-only training set.
Experimental setup
Datasets: DSEC-Detection MVSEC
Benchmarks: DSEC-Detection MVSEC
Metrics: mAP mAP50 Abs.Rel. RMS RMSlog Params Runtime
Baselines: Faster R-CNN YOLOX Swin Transformer Deformable DETR EMS-YOLO AsyNet AEGNN ASTM-Net FPN-fusion SFNet DAGr SODFormer
Builds on: DAGr YOLOX SODFormer Lite-Mono deformable convolution gradient reversal layer
Closest papers by meaning
- Multimodal Event Causality Reasoning with Scene Graph Enhanced Interaction Network (AAAI 2024 · similarity 0.86)
- Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation (CVPR 2025 · similarity 0.86)
- DiffVsgg: Diffusion-Driven Online Video Scene Graph Generation (CVPR 2025 · similarity 0.86)
- LASER: A Neuro-Symbolic Framework for Learning Spatio-Temporal Scene Graphs with Weak Supervision (ICLR 2025 · similarity 0.85)
- Graph Neural Network Combining Event Stream and Periodic Aggregation for Low-Latency Event-based Vision (CVPR 2025 · similarity 0.85)
- Universal Scene Graph Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.