Research map / Architectures and efficiency / RNNs and normalization
Graph Neural Network Combining Event Stream and Periodic Aggregation for Low-Latency Event-based Vision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HUGNet2+PA, an accumulation-free asynchronous event-graph GNN combined with a periodic aggregator branch, for ultra-low-latency event-based optical flow estimation.
Problem
Event-based optical flow estimation faces an accuracy-latency trade-off: frame-based CNNs add tens of milliseconds of latency, while asynchronous event graph GNNs without temporal accumulation lose global context and accuracy.
Approach
Combine an asynchronous, accumulation-free event-graph branch (HUGNet2, with no normalization or normal vectors) with a periodic branch that max-pools GNN features over past periods and runs convolutional and recurrent aggregation; the event head merges GNN features with the periodic output from two periods earlier, adding no latency.
Key results
- On Rock Scenes, HUGNet2+PA100 reaches endpoint error 0.030 versus 0.037 for HUGNet and 0.048 for ADMFlow at 20 Hz, with about 50 µs estimated latency.
- On MVSEC, HUGNet2+PA20 achieves endpoint error 1.52 with about 48x fewer operations per second than ADMFlow, at an estimated latency of about 50 µs versus 50-100 ms for frame-based methods.
- Event prediction with low PA rate reduces error by up to 59% compared to the periodic prediction from two periods earlier, and detects abrupt motion changes faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add periodic aggregation to HUGNet2 | HUGNet2 · MVSEC | Endpoint Error: 1.89 -> 1.52 | Helped |
| Add periodic aggregation to HUGNet2 | HUGNet2 · Rock Scenes | Endpoint Error: 0.066 -> 0.030 | Helped |
| Replace HUGNet with accumulation-free HUGNet2 | HUGNet · MVSEC | Endpoint Error: 1.72a -> 1.89 | Hurt |
| Replace HUGNet with accumulation-free HUGNet2 | HUGNet · Rock Scenes | Endpoint Error: 0.037a -> 0.066 | Hurt |
| Increase periodic aggregation rate from 5 Hz to 20 Hz (ablation) | HUGNet2+PA · MVSEC | Endpoint Error: 1.71 -> 1.52 | Helped |
| Increase periodic aggregation rate from 20 Hz to 100 Hz (ablation) | HUGNet2+PA · Rock Scenes | Endpoint Error: 0.040 -> 0.030 | Helped |
| Use Event prediction instead of Periodic(T-2) prediction | HUGNet2+PA · Rock Scenes | Endpoint Error: -59% | Helped |
| Use Event prediction instead of Periodic(T-2) prediction | HUGNet2+PA · MVSEC | Endpoint Error: -29% | Helped |
| Use the Event prediction rather than a periodic output to detect abrupt motion changes | HUGNet2+PA20 · Rock Scenes | motion-change detection latency: 50 to 100 ms | Helped |
| Add periodic aggregation to HUGNet2 | HUGNet2 · MVSEC | OPs/s: 13.7 G -> 19.8 G | Hurt |
Surprising findings
- HUGNet2+PA with a low periodic aggregation rate matches the accuracy of the periodic prediction at a much higher rate, while using fewer operations.
Limitations
- On MVSEC, accuracy remains below state-of-the-art frame-based methods such as ADMFlow.
- Latency on asynchronous hardware is estimated from a prior FPGA implementation rather than measured.
- Evaluated only on MVSEC and the synthetic Rock Scenes dataset; event-level prediction is sparse and temporally shifted by the periodic delay.
Experimental setup
Datasets: MVSEC Rock Scenes
Benchmarks: MVSEC Rock Scenes
Metrics: sparse endpoint error parameters operations per second latency
Baselines: HUGNet ADMFlow EV-FlowNet E-RAFT STE-FlowNet Spike-FlowNet SdformerFlow FireFlowNet HUGNet2
Builds on: HUGNet E-RAFT PointTransformerConv GCNConv ADMFlow
Closest papers by meaning
- EventNet: Asynchronous Recursive Event Processing (CVPR 2019 · similarity 0.90)
- AEGNN: Asynchronous Event-Based Graph Neural Networks (CVPR 2022 · similarity 0.89)
- Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing (AAAI 2025 · similarity 0.89)
- EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation (CVPR 2025 · similarity 0.88)
- Asynchronous Collaborative Graph Representation for Frames and Events (CVPR 2025 · similarity 0.85)
- Efficient Meshflow and Optical Flow Estimation from Event Cameras (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.