Research map / Video / Video segmentation and tracking
EventMamba: Enhancing Spatio-Temporal Locality with State Space Models for Event-Based Video Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EventMamba, a U-Net-like vision Mamba network for event-based video reconstruction that uses random window offsets in space and Hilbert/trans-Hilbert space-filling-curve scanning in spatio-temporal space.
Problem
Event-based video reconstruction needs spatial translation invariance and preservation of local spatio-temporal event relations, which fixed-window and reshape-scan vision Mamba models break, while Transformers are quadratic in input size for high-resolution event data.
Approach
Replace fixed window partitions with random window offsets during training (Monte Carlo averaging at test time) for translation invariance, and serialize spatio-temporal event features with Hilbert and trans-Hilbert space-filling curves so adjacent events remain adjacent in the sequence.
Key results
- On IJRR, EventMamba achieves MSE 0.039 and SSIM 0.586, best among compared methods; the paper states a 2.9% SSIM gain over prior state of the art.
- On MVSEC, EventMamba reports MSE 0.073, SSIM 0.328, LPIPS 0.475, the best on all three metrics.
- At 1280x720, EventMamba inference takes 136.14 ms with 11.21M parameters, versus 1.85 s for ET-Net, per the paper's table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EventMamba's RWO and HSFC design | EventMamba · IJRR | MSE: 0.048 -> 0.039 | Helped |
| add EventMamba's RWO and HSFC design | EventMamba · IJRR | SSIM: 0.562 -> 0.586 | Helped |
| replace fixed windows with random window offsets (ablation) | EventMamba with Hilbert scanning · IJRR | MSE: 0.048 -> 0.039 | Helped |
| replace fixed windows with random window offsets (ablation) | EventMamba with Hilbert scanning · IJRR | SSIM: 0.562 -> 0.586 | Helped |
| use both Hilbert and Trans-Hilbert scans instead of Hilbert alone (ablation) | EventMamba · IJRR | MSE: 0.041 -> 0.039 | Helped |
| use both Hilbert and Trans-Hilbert scans instead of Hilbert alone (ablation) | EventMamba · IJRR | SSIM: 0.582 -> 0.586 | Helped |
| use both Hilbert and Trans-Hilbert scans instead of Trans-Hilbert alone (ablation) | EventMamba · IJRR | MSE: 0.041 -> 0.039 | Helped |
| use both Hilbert and Trans-Hilbert scans instead of Trans-Hilbert alone (ablation) | EventMamba · IJRR | SSIM: 0.581 -> 0.586 | Helped |
| replace PyTorch reshape scanning with HSFC (ablation) | EventMamba with RWO · IJRR | MSE: 0.045 -> 0.039 | Helped |
| replace PyTorch reshape scanning with HSFC (ablation) | EventMamba with RWO · IJRR | SSIM: 0.574 -> 0.586 | Helped |
| increase base channels from 32 to 48 (ablation) | EventMamba · IJRR | MSE: 0.039 -> 0.039 | No clear effect |
| increase base channels from 32 to 48 (ablation) | EventMamba · IJRR | SSIM: 0.586 -> 0.587 | Helped |
Surprising findings
- At test time, the paper replaces exact expectation over random windows with Monte Carlo averaging over M=8 offsets, which it says keeps inference cost manageable.
Limitations
- The paper reports a slight LPIPS decrease versus E2VID+ on HQF.
- Test-time Monte Carlo averaging over random windows adds inference cost relative to a single deterministic pass.
Experimental setup
Datasets: ESIM-rendered synthetic multi-object sequences (MS-COCO objects) HQF IJRR MVSEC
Benchmarks: HQF IJRR MVSEC
Metrics: MSE SSIM LPIPS Params (M) inference time (ms)
Baselines: FireNet+ E2VID+ ET-Net SPADE-E2VID EVSNN HyperE2VID
Builds on: Mamba VisionMamba E2VID LPIPS Hilbert space-filling curve scanning
Closest papers by meaning
- SMamba: Sparse Mamba for Event-based Object Detection (AAAI 2025 · similarity 0.88)
- Event-based Video Super-Resolution via State Space Models (CVPR 2025 · similarity 0.88)
- LC-Mamba: Local and Continuous Mamba with Shifted Windows for Frame Interpolation (CVPR 2025 · similarity 0.87)
- LaSe-E2V: Towards Language-guided Semantic-aware Event-to-Video Reconstruction (NeurIPS 2024 · similarity 0.86)
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.86)
- EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.