Research map / Video / Video segmentation and tracking
Learning Scale-Aware Spatio-temporal Implicit Representation for Event-based Motion Deblurring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SASNet, a scale-aware spatio-temporal implicit representation network that deblurs images using events at arbitrary spatial and temporal scales, and releases the H2D real dataset captured with a hybrid EVS/CIS sensor.
Problem
Event-based motion deblurring assumes images and events share the same spatial and temporal scales, which fails in practice when scales differ; existing datasets are also low-resolution or misaligned.
Approach
Implicitly aggregate spatial correspondence (via event encoding sampling and an MLP-style implicit decoder, SIRM) and temporal correspondence (via time-aware event selection and temporal grouped shift, TIRM) so one model generalizes across continuous spatial and temporal scales.
Key results
- SASNet reaches 28.82 dB PSNR / 0.8811 SSIM on GoPro (Rs=4, Rt=1) and 35.72 dB / 0.9541 on H2D, outperforming EFNet (28.08 / 34.59 dB).
- With 1.46M parameters and 43.35G FLOPs, SASNet is much smaller than Restormer (26.13M) and REFID (88.96M).
- Under increasing blur frames (3 to 9), SASNet drops only 0.29 dB while EFNet drops 0.73 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SIRM and TIRM (ablation) | SASNet · GoPro | PSNR: 26.16 -> 28.82 | Helped |
| add SIRM while retaining TIRM (ablation) | SASNet · GoPro | PSNR: 28.28 -> 28.82 | Helped |
| add TIRM while retaining SIRM (ablation) | SASNet · GoPro | PSNR: 26.98 -> 28.82 | Helped |
| replace SIRM with interpolation (ablation) | SASNet · GoPro | PSNR: 28.36 vs 28.82 | Hurt |
| replace SIRM with transposed convolution (ablation) | SASNet · GoPro | PSNR: 28.12 vs 28.82 | Hurt |
| replace SIRM with pixel shuffle (ablation) | SASNet · GoPro | PSNR: 28.43 vs 28.82 | Hurt |
| replace SIRM with learnable upsample (ablation) | SASNet · GoPro | PSNR: 28.61 vs 28.82 | Hurt |
| replace TIRM with convolution (ablation) | SASNet · GoPro | PSNR: 27.72 vs 28.82 | Hurt |
| replace TIRM with optical flow (ablation) | SASNet · GoPro | PSNR: 27.99 vs 28.82 | Hurt |
| replace TIRM with deformable convolution (ablation) | SASNet · GoPro | PSNR: 28.26 vs 28.82 | Hurt |
| replace TIRM with VIT self-attention (ablation) | SASNet · GoPro | PSNR: 28.84 vs 28.82 | No clear effect |
| increase event voxel grid bins from 8 to 32 (ablation) | SASNet · GoPro | PSNR: 25.43 -> 28.97 | Helped |
Surprising findings
- On H2D, RED-Net (33.98 dB) and EFNet (34.59 dB) outperform the frame-based Restormer by a wide margin, while REFID underperforms on H2D despite its large size.
Limitations
- Robustness to real-world degradations beyond motion blur is not addressed.
- Scale ranges are finite (spatial 1 to 4, temporal (0,1]) in the experiments.
Experimental setup
Datasets: GoPro H2D
Benchmarks: GoPro H2D
Metrics: PSNR SSIM #Params #FLOPs
Baselines: HINet NAFNet Restormer RED-Net EVDI UEVD EFNet REFID
Builds on: EFNet REFID EDI model Restormer
Closest papers by meaning
- Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events (AAAI 2024 · similarity 0.90)
- Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · similarity 0.90)
- Motion-adaptive Transformer for Event-based Image Deblurring (AAAI 2025 · similarity 0.89)
- Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · similarity 0.87)
- Event-Enhanced Blurry Video Super-Resolution (AAAI 2025 · similarity 0.86)
- Asymmetric Hierarchical Difference-aware Interaction Network for Event-guided Motion Deblurring (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.