Research map / Image generation / Image restoration
A Polarization-Aided Transformer for Image Deblurring via Motion Vector Decomposition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MDT, a transformer for image deblurring that decomposes motion into translational (Cartesian deformable conv) and rotational (polar-offset deformable conv) components, with a radial-stripe polar attention (RSAS) using angular relative positional encoding.
Problem
Image deblurring methods model blur along orthogonal Cartesian directions and handle translational and rotational motion jointly, leading to limited performance on complex, rotation-dominated blur.
Approach
Separate the motion into translation and rotation via a dual-branch motion decomposition module (Cartesian and polar deformable convolutions), and replace Cartesian window attention with radial stripe windows plus angular relative positional encoding in the polar system.
Key results
- MDT reaches 34.26 dB PSNR on GoPro and 31.84 dB on HIDE, outperforming FFTformer (34.21 / 31.62) with 14.3M vs 16.6M parameters.
- On REDS-val-300, MDT achieves 29.25 dB PSNR, 0.11 dB above FFTformer (29.14) with fewer parameters.
- Ablation on GoPro: RSAS gives a 1.85 dB PSNR gain over the CNN+FSAS baseline; MDM gives 0.86 dB gain over CNN embedding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CNN embedding with the dual-branch MDM (ablation) | MDT · GoPro | PSNR: 31.00 -> 31.86 | Helped |
| replace FSAS with RSAS (ablation) | MDT with MDM · GoPro | PSNR: 30.60 -> 31.86 | Helped |
| apply MDM at multiple scales instead of only once (ablation) | MDT · GoPro | PSNR: 31.86 -> 30.22 | Hurt |
| run the Transformer block in parallel with MDM (ablation) | MDT · GoPro | PSNR: 31.86 -> 27.96 | Hurt |
| add translation and rotation offsets before deformable convolution instead of learning motion components separately (ablation) | MDT · GoPro | PSNR: 31.86 -> 31.32 | Hurt |
| use polar-coordinate MDT for rotational rather than Cartesian FFTformer processing | image deblurring · Rotation blur | PSNR: 27.59 -> 28.10 | Helped |
| use polar-coordinate MDT rather than Cartesian FFTformer processing | image deblurring · Translation blur | PSNR: 31.73 -> 31.68 | Hurt |
Surprising findings
- Using MDM repeatedly in deeper layers hurts performance, since clean content gets treated as blur (Only-once design is best).
Limitations
- Limited performance on defocus blur, which lacks motion structure (e.g., DDPD defocus gain only 0.07 dB).
- Rotation-subset evaluation uses a manually curated subset of existing data rather than a new benchmark.
Experimental setup
Datasets: GoPro REDS RSBlur
Benchmarks: GoPro HIDE RealBlur-R RealBlur-J RSBlur REDS-val-300 RWBI DDPD
Metrics: PSNR SSIM NIQE LPIPS Params GMACs
Baselines: DeblurGAN-v2 DMPHN MIMO MPRNet Restormer Uformer Stripformer SFNet FFTformer UFPNet MISC Filter FPro
Builds on: Stripformer FFTformer Uformer DCNv2 Restormer
Closest papers by meaning
- Real-World Efficient Blind Motion Deblurring via Blur Pixel Discretization (CVPR 2024 · similarity 0.79)
- Adaptive Window Pruning for Efficient Local Motion Deblurring (ICLR 2024 · similarity 0.79)
- Quality-Improved and Property-Preserved Polarimetric Imaging via Complementarily Fusing (NeurIPS 2024 · similarity 0.78)
- ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration (NeurIPS 2024 · similarity 0.78)
- Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring (CVPR 2024 · similarity 0.78)
- Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.