Research map / Video / Video segmentation and tracking
Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STCNet, a spatial-temporal collaboration network that fuses blurry frames and event streams for event-based motion deblurring, using differential-modality cross-modal calibration, co-attention fusion, and frame-event mutual spatio-temporal attention.
Problem
Event-based motion deblurring fails to handle modality redundancy in spatial fusion and does not exploit temporal cooperation between events and frames, especially under severe or fast motion blur.
Approach
Calibrates frame and event features using a differential-modality (feature subtraction) guided cross-attention, balances modality contributions with cross-modal co-attention, and models cross-temporal dependencies with a frame-event mutual spatio-temporal attention aggregated via a cross-temporal coordinate attention.
Key results
- On GoPro, STCNet reaches 36.45 dB PSNR / 0.9809 SSIM, 0.54 dB above the best event-based baseline (EFNet 35.46 dB... REFID 35.91 dB).
- On DVD (trained on GoPro), STCNet reaches 33.94 dB PSNR, about 0.79 dB above the best event-based method REFID (33.15 dB).
- On the real REB dataset, STCNet reaches 35.44 dB PSNR, 0.53 dB above the best event-based method EFNet (34.91 dB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STCNet instead of REFID | Event-based motion deblurring · GoPro | PSNR: 35.91 -> 36.45 | Helped |
| Use STCNet instead of REFID | Event-based motion deblurring · DVD | PSNR: 33.15 -> 33.94 | Helped |
| Use STCNet instead of EFNet | Event-based motion deblurring · REB | PSNR: 34.91 -> 35.44 | Helped |
| Add MSC (ablation) | STCNet baseline · GoPro | PSNR: 33.40 -> 35.93 | Helped |
| Add MSC (ablation) | STCNet baseline · REB | PSNR: 33.12 -> 35.00 | Helped |
| Add MTC (ablation) | STCNet baseline · GoPro | PSNR: 33.40 -> 35.05 | Helped |
| Add MTC (ablation) | STCNet baseline · REB | PSNR: 33.12 -> 34.20 | Helped |
| Add mutual cross-frame and cross-event attention (ablation) | MTC · GoPro | PSNR: 33.40 -> 34.61 | Helped |
| Add mutual cross-frame and cross-event attention (ablation) | MTC · REB | PSNR: 33.12 -> 33.79 | Helped |
| Add DM-CMC and CMA (ablation) | MSC · GoPro | PSNR: 33.40 -> 35.93 | Helped |
| Add DM-CMC and CMA (ablation) | MSC · REB | PSNR: 33.12 -> 35.00 | Helped |
| Add CTF (ablation) | MTC · GoPro | PSNR: 34.61 -> 35.05 | Helped |
Limitations
- Relies on synthetic event simulation (ESIM) for GoPro/DVD training, with real-event evaluation limited to the authors' REB dataset
- Event-based approach requires event camera hardware, limiting applicability to standard cameras
Experimental setup
Datasets: GoPro DVD REB
Benchmarks: GoPro DVD REB
Metrics: PSNR SSIM Params (M) Runtime (s)
Baselines: MemDeblur MMP-RNN MPRNet MIMO-UNet++ Restormer RNN-MBP NAFNet VRT DFFN DSTN RED eSL-Net
Builds on: EFNet Coordinate attention Transformer self-attention Charbonnier loss
Closest papers by meaning
- Learning Scale-Aware Spatio-temporal Implicit Representation for Event-based Motion Deblurring (ICML 2024 · similarity 0.90)
- Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · similarity 0.89)
- Asymmetric Hierarchical Difference-aware Interaction Network for Event-guided Motion Deblurring (AAAI 2025 · similarity 0.89)
- Motion-adaptive Transformer for Event-based Image Deblurring (AAAI 2025 · similarity 0.89)
- Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow (CVPR 2025 · similarity 0.89)
- Diffusion-based Event Generation for High-Quality Image Deblurring (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.