Research map / Architectures and efficiency / Attention and transformers
Motion-adaptive Transformer for Event-based Image Deblurring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAT, a motion-adaptive Transformer for event-based image deblurring that predicts motion masks from event features to guide sparse and motion-aware attention, achieving SOTA on GoPro, HS-ERGB, REBlur and REVD.
Problem
Event-based image deblurring methods mostly use event intensity and CNNs, ignoring spatial motion information and failing to model non-local dependencies; standard Transformer attention cannot discriminate relevant tokens in sparse events and dense images.
Approach
An Adaptive Motion Mask Predictor (AMMP) derives a top-k motion mask from event features that guides Motion-Sparse Attention on events and Motion-Aware Attention on images; Expansion-Controlled Spatial Gating makes the non-differentiable top-k selection learnable, and Cross-Modal Intensity Gating fuses event intensity into image features.
Key results
- Achieves 36.67 dB PSNR on GoPro, up to 0.60 dB over MAENet (36.07 dB).
- Achieves 28.97 dB PSNR on HS-ERGB, 1.04 dB over MAENet.
- On REVD, reaches 33.05 dB PSNR, 0.82 dB above MAENet's 32.23 dB, and 0.06 dB above video method FEVD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAT instead of MAENet | Event-based deblurring · GoPro test | PSNR: 36.07 -> 36.67 | Helped |
| Use MAT instead of MAENet | Event-based deblurring · HS-ERGB test | PSNR: 27.93 -> 28.97 | Helped |
| Use MAT instead of MAENet | Event-based deblurring · REBlur test | PSNR: 38.47 -> 38.69 | Helped |
| Use MAT instead of FEVD | Event-based deblurring · REVD test | PSNR: 32.99 -> 33.05 | Helped |
| Add AMMP in place of the Top-k mask predictor (ablation) | MAT with MSA and MAA on GoPro · GoPro | PSNR: 34.82 -> 35.23 (+0.41dB) | Helped |
| Add MSA and MAA instead of standard attention without mask guidance (ablation) | MAT with AMMP on GoPro · GoPro | PSNR: 34.47 -> 35.23 | Helped |
| Add CMIG instead of simple addition fusion (ablation) | MAT on GoPro · GoPro | PSNR: 34.19 -> 34.47 | Helped |
| Use CMIG instead of concatenation fusion (ablation) | MAT on GoPro · GoPro | PSNR: 34.31 -> 35.23 | Helped |
| Use CMIG instead of cross-attention fusion (ablation) | MAT on GoPro · GoPro | PSNR: 34.84 -> 35.23 | Helped |
| Add MAA alongside MSA (ablation) | MAT with AMMP on GoPro · GoPro | PSNR: 34.85 -> 35.23 | Helped |
| Add MSA alongside MAA (ablation) | MAT with AMMP on GoPro · GoPro | PSNR: 34.64 -> 35.23 | Helped |
| Change the event representation among SBT, SCER, and DA (ablation) | MAT on GoPro · GoPro | PSNR: 36.62; 36.67; 36.46 | Mixed |
Surprising findings
- Simply concatenating events to input lets Restormer reach performance comparable to event-based CNN methods, indicating CNN local receptive fields limit event-based methods.
Limitations
- Relies on event voxel bins and a sparsity-based rule for choosing k; the expansion factor is a heuristic
- Results on REVD use single image plus 6 event bins, not directly comparable to video methods with more frames
- Evaluated only on image deblurring datasets with synthetic or limited real event data
Experimental setup
Datasets: GoPro HS-ERGB REBlur REVD
Benchmarks: GoPro HS-ERGB REBlur REVD
Metrics: PSNR SSIM FLOPs parameters
Baselines: EDI EVDI HINet MSDI-Net NAFNet UFPNet EFNet EIFNet MAENet Restormer FFTFormer FEVD
Builds on: Restormer EFNet MAENet transposed attention
Closest papers by meaning
- Learning Scale-Aware Spatio-temporal Implicit Representation for Event-based Motion Deblurring (ICML 2024 · similarity 0.89)
- Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events (AAAI 2024 · similarity 0.89)
- Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · similarity 0.88)
- Asymmetric Hierarchical Difference-aware Interaction Network for Event-guided Motion Deblurring (AAAI 2025 · similarity 0.88)
- Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · similarity 0.88)
- Diffusion-based Event Generation for High-Quality Image Deblurring (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.