Research map / Video / Video segmentation and tracking
Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BSSTNet, a video deblurring network that uses parameter-free blur maps from optical flow to sparsify spatio-temporal attention over longer windows and to guide bidirectional feature propagation.
Problem
Video deblurring needs information from distant frames, but dense spatio-temporal self-attention limits temporal window length due to memory and compute, and bidirectional flow-based propagation accumulates errors from inaccurate flow in blurry frames.
Approach
Derive a non-learnable blur map from optical flow magnitudes and use it to (1) keep only blurry query windows and sharp key/value windows in a spatio-temporal transformer, enabling longer temporal windows at lower cost, and (2) gate deformable-convolution alignment in bidirectional propagation with sharp maps so blurry pixels are not propagated.
Key results
- On DVD, BSSTNet reaches 34.95 dB PSNR and 0.9703 SSIM, 0.26 dB above Shift-Net+ (34.69).
- On GoPro, BSSTNet reaches 35.98 dB PSNR and 0.9792 SSIM, slightly above Shift-Net+ (35.88 dB).
- BSST at temporal length 48 uses 133 GFLOPs vs 329 for standard spatio-temporal transformer, while standard transformer runs out of memory at length 60.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BSSTNet instead of Shift-Net+ | Video deblurring · GoPro | PSNR: 35.88 -> 35.98 | Helped |
| use BSSTNet instead of Shift-Net+ | Video deblurring · GoPro | SSIM: 0.9790 -> 0.9792 | Helped |
| use BSSTNet instead of Shift-Net+ | Video deblurring · DVD | PSNR: 34.69 -> 34.95 | Helped |
| use BSSTNet instead of Shift-Net+ | Video deblurring · DVD | SSIM: 0.9690 -> 0.9703 | Helped |
| use BSSTNet instead of Shift-Net+ | Video deblurring · DVD (256 × 256 frame) | GFLOPs: 146 -> 133 | Helped |
| remove BBFP (ablation) | BSSTNet · DVD | PSNR: 34.95 -> 34.74 | Hurt |
| remove BBFP (ablation) | BSSTNet · DVD | SSIM: 0.9703 -> 0.9692 | Hurt |
| remove BSST (ablation) | BSSTNet · DVD | PSNR: 34.95 -> 34.10 | Hurt |
| remove BSST (ablation) | BSSTNet · DVD | SSIM: 0.9703 -> 0.9661 | Hurt |
| replace BFA with standard flow-guided feature alignment (ablation) | BSSTNet · DVD | PSNR: 34.95 -> 34.82 | Hurt |
| replace BFA with standard flow-guided feature alignment (ablation) | BSSTNet · DVD | SSIM: 0.9703 -> 0.9696 | Hurt |
| select tokens using the blur-map-guided Top 50% strategy rather than random 50% (ablation) | BSSTNet · DVD | PSNR: 33.92 -> 34.95 | Helped |
Surprising findings
- Keeping only the top 25% of blur-selected tokens gives nearly the same PSNR as using all tokens (34.78 vs 34.98 dB) at about 43% of the FLOPs.
Limitations
- Blur maps are derived from optical flow magnitude, so their quality depends on the flow estimator (fixed pre-trained RAFT).
- Evaluation is limited to two datasets (GoPro and DVD) and PSNR/SSIM metrics.
Experimental setup
Datasets: GoPro DVD
Benchmarks: GoPro DVD
Metrics: PSNR SSIM GFLOPs runtime (ms) GPU memory (GB)
Baselines: STFAN STDAN RNN-MBP NAFNet VRT RVRT Shift-Net+ ARVo
Builds on: VRT RVRT BasicVSR++ RAFT Swin-style window attention
Closest papers by meaning
- Motion-adaptive Transformer for Event-based Image Deblurring (AAAI 2025 · similarity 0.88)
- Bidirectional Sparse Attention for Faster Video Diffusion Training (NeurIPS 2026 · similarity 0.87)
- Learning Scale-Aware Spatio-temporal Implicit Representation for Event-based Motion Deblurring (ICML 2024 · similarity 0.87)
- Blur2Blur: Blur Conversion for Unsupervised Image Deblurring on Unknown Domains (CVPR 2024 · similarity 0.86)
- BlurDM: A Blur Diffusion Model for Image Deblurring (NeurIPS 2025 · similarity 0.86)
- Event-Enhanced Blurry Video Super-Resolution (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.