Research map / Video / Video segmentation and tracking
Event-based Video Super-Resolution via State Space Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MamEVSR, a bidirectional recurrent Mamba-based network for event-based video super-resolution that fuses RGB frames and event voxels via interleaved (iMamba) and cross-modality (cMamba) selective SSM blocks.
Problem
Event-based video super-resolution needs effective temporal feature propagation and cross-modal RGB-event fusion, but CNN and attention methods have limited receptive fields and high computational cost at high resolution.
Approach
Interleave tokens from adjacent frames into one sequence and scan with multi-directional selective SSMs (iMamba) for linear-complexity inter-frame modeling, plus a cross-selective-SSM block (cMamba) that exchanges information between RGB and event branches.
Key results
- On CED x2, MamEVSR reaches 41.14 dB PSNR / 0.9831 SSIM in the scene-averaged table (33.74 dB in text), beating EvTexture+ at 40.57 dB in the table.
- On REDS4 x4, MamEVSR averages 32.94 dB, slightly above EvTexture+ (32.94 in table) with 9.6M vs 10.1M parameters.
- Ablation on CED x2 shows removing the iMamba block drops PSNR from 41.14 to 40.17 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MamEVSR instead of EvTexture+ | Event-based VSR · CED ×2 | PSNR: 40.57 -> 41.14 | Helped |
| Use MamEVSR instead of EvTexture+ | Event-based VSR · CED ×2 | SSIM: 0.9815 -> 0.9831 | Helped |
| Use MamEVSR instead of EvTexture+ | Event-based VSR · REDS4 ×4 average | PSNR: 32.93 -> 32.94 | Helped |
| Use MamEVSR instead of EvTexture+ | Event-based VSR · REDS4 ×4 average | SSIM: 0.9195 -> 0.9195 | No clear effect |
| Add the iMamba block to the core modules (ablation) | MamEVSR · CED ×2 | PSNR: 40.17 -> 41.14 | Helped |
| Add the cMamba block (ablation) | MamEVSR · CED ×2 | PSNR: 40.26 -> 41.14 | Helped |
| Add the forward and backward recurrent cells (ablation) | MamEVSR · CED ×2 | PSNR: 40.07 -> 41.14; 40.08 -> 41.14 | Helped |
| Use the iMamba block rather than residual blocks (ablation) | MamEVSR · CED ×2 | PSNR: 40.62 -> 41.14 | Helped |
| Use the iMamba block rather than deformable convolution (ablation) | MamEVSR · CED ×2 | PSNR: 40.72 -> 41.14 | Helped |
| Use the iMamba block rather than flow warping (ablation) | MamEVSR · CED ×2 | PSNR: 40.84 -> 41.14 | Helped |
| Add interleaved reorganization to iMamba (ablation) | MamEVSR · CED ×2 | PSNR: 40.99 -> 41.14 | Helped |
| Add channel attention to iMamba (ablation) | MamEVSR · CED ×2 | PSNR: 41.02 -> 41.14 | Helped |
Surprising findings
- Removing the iMamba block from the reconstructor slightly improves PSNR and SSIM, and cMamba removal has only a small effect.
Limitations
- Struggles to recover fine textures due to information loss in RGB and low-resolution, poorly defined event data.
- Uses only short temporal windows; longer sequences are suggested as future improvement.
Experimental setup
Datasets: REDS CED ESIM-simulated events
Benchmarks: REDS4 CED
Metrics: PSNR SSIM parameters runtime
Baselines: EvTexture EvTexture+ EBVSR EGVSR E-VSR BasicVSR RBPN TDAN EDVR VRT TTVSR DUF
Builds on: Mamba BasicVSR MambaIR EvTexture
Closest papers by meaning
- Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution (CVPR 2023 · similarity 0.89)
- EventMamba: Enhancing Spatio-Temporal Locality with State Space Models for Event-Based Video Reconstruction (AAAI 2025 · similarity 0.88)
- Self-supervised ControlNet with Spatio-Temporal Mamba for Real-world Video Super-resolution (CVPR 2025 · similarity 0.87)
- EvEnhancer: Empowering Effectiveness, Efficiency and Generalizability for Continuous Space-Time Video Super-Resolution with Events (CVPR 2025 · similarity 0.87)
- EvSTVSR: Event Guided Space-Time Video Super-Resolution (AAAI 2025 · similarity 0.87)
- Event-Enhanced Blurry Video Super-Resolution (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.