Research map / Video / Video segmentation and tracking
Bilateral Event Mining and Complementary for Event Stream Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BMCNet, a two-stream network that separately mines positive and negative event streams and exchanges global structure between them via a bilateral information exchange module for event stream super-resolution.
Problem
Event stream super-resolution (ESR) suffers from low spatial resolution; prior learning-based methods process positive and negative events in a mixed paradigm, failing to model each polarity's distinct distribution and their complementary correlations.
Approach
Decouple positive and negative events into two parallel streams and exchange global structural information between them through a channel-wise cross-attention bilateral information exchange (BIE) module with cross-layer interaction representations, plus an inner-stream BIE for spatial-temporal context.
Key results
- BMCNet achieves about 11% average RMSE improvement over RecEvSR across NFS-syn, RGB-syn and EventNFS-real (e.g., 0.552 vs 0.576 on NFS-syn 8x... 4x RMSE 0.552 vs 0.607).
- BMCNet-plain reduces parameters by 40% vs RecEvSR while improving RMSE by about 7.5% on average.
- On NCars object recognition at 2x, BMCNet reaches 68.58 ACC vs 62.91 for RecEvSR; video reconstruction SSIM also improves.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BMCNET instead of RecEvSR | Event stream super-resolution · NFS-syn 2× | RMSE: 0.656 -> 0.564 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · NFS-syn 4× | RMSE: 0.607 -> 0.552 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · NFS-syn 8× | RMSE: 0.576 -> 0.553 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · RGB-syn 2× | RMSE: 0.319 -> 0.276 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · RGB-syn 4× | RMSE: 0.296 -> 0.274 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · EventNFS-real 2× | RMSE: 0.613 -> 0.527 | Helped |
| Use BMCNET instead of RecEvSR | Event stream super-resolution · EventNFS-real 4× | RMSE: 0.670 -> 0.625 | Helped |
| Add a recurrent hidden state (ablation) | BMCNET ablation model · NFS-syn 4× | RMSE: 0.605 -> 0.599 | Helped |
| Add a recurrent hidden state (ablation) | BMCNET ablation model · EventNFS-real 4× | RMSE: 0.655 -> 0.648 | Helped |
| Add event decoupling and recurrent state without BIE (ablation) | BMCNET ablation model · NFS-syn 4× | RMSE: 0.605 -> 0.637 | Hurt |
| Add event decoupling and recurrent state without BIE (ablation) | BMCNET ablation model · EventNFS-real 4× | RMSE: 0.655 -> 0.768 | Hurt |
| Add inner-stream BIE for spatial and temporal context modeling (ablation) | BMCNET with inter-stream BIE and recurrent state · NFS-syn 4× | RMSE: 0.577 -> 0.552 | Helped |
Surprising findings
- Mixed-polarity processing produces overlapped artifacts that disappear when positive and negative events are decoupled.
- Cross-attention over spatial positions is more easily misled by noise than the proposed channel-wise exchange, performing worse than BIE.
Limitations
- Real-world high-factor ESR datasets are scarce; real evaluation is limited to 2x and 4x on EventNFS.
- Full BMCNet has 50% more parameters than RecEvSR and higher FLOPs at 8x on real data.
Experimental setup
Datasets: EventNFS NFS-syn RGB-syn NCars NFS
Benchmarks: NFS-syn RGB-syn EventNFS-real NCars
Metrics: RMSE parameters FLOPs ACC AUC SSIM LPIPS
Baselines: bicubic SRFBN RLSP RSTT EventZoom RecEvSR
Builds on: RecEvSR EventZoom E2VID Scaled Dot-Product Attention
Closest papers by meaning
- LaSe-E2V: Towards Language-guided Semantic-aware Event-to-Video Reconstruction (NeurIPS 2024 · similarity 0.85)
- E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning (AAAI 2024 · similarity 0.84)
- Flow-Event Autoencoder: Event Stream Object Recognition Dataset Generation with Arbitrary High Temporal Resolution (AAAI 2024 · similarity 0.84)
- OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies (CVPR 2024 · similarity 0.84)
- Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline (CVPR 2024 · similarity 0.84)
- EventGPT: Event Stream Understanding with Multimodal Large Language Models (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.