Research map / Image generation / Image restoration
Improving the Learning Capability of Small-size Image Restoration Network by Deep Fourier Shifting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Fourier Shifting, an information-lossless shift operator implemented as Fourier-domain cycling (amplitude-phase and real-imaginary variants) that replaces standard convolutions in small image restoration networks.
Problem
Spatial shift operators, effective for high-level vision, lose region-aware information through zero-filling, which hurts low-level image restoration tasks such as denoising, enhancement, deblurring and guided super-resolution.
Approach
Apply the spatial shift in the Fourier domain, where the DFT is periodic so shifting becomes a lossless circular roll (Fourier cycling) of amplitude/phase or real/imaginary components, followed by 1x1 convs and inverse FFT, as a drop-in replacement for standard convolution units.
Key results
- On LOL, replacing SID's convolutions with Fcycle-AP raised PSNR from 20.11 to 22.86 dB while reducing parameters from 7.76M to 7.53M.
- On SIDD denoising with DnCNN, Fcycle-ab reached 38.18 dB PSNR versus 37.20 dB for the original, with parameters reduced from 1.51M to 1.43M.
- Spatial shift (Shift-sa) lowered SID's PSNR on LOL slightly (20.01 vs 20.11 dB) while Fourier variants improved it, and Shift-sa at shift 18 collapsed DnCNN to 23.69 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard convolution with amplitude-phase Deep Fourier Shifting | DRBN · LOL→LOL | PSNR: 19.7931 -> 22.4274 | Helped |
| Replace standard convolution with real-imaginary Deep Fourier Shifting | DnCNN · SIDD | PSNR: 37.1992 -> 38.1837 | Helped |
| Replace standard convolution with amplitude-phase Deep Fourier Shifting | DeepDeblur · GoPro | PSNR: 28.9423 -> 29.2123 | Helped |
| Replace standard convolution with real-imaginary Deep Fourier Shifting | PANNET · WorldView-III | PSNR: 29.68 -> 30.55 | Helped |
| Replace standard convolution with spatial shifting (Shift-sa) | SID · LOL and Huawei enhancement | PSNR: The paper reports reductions of 0.1dB/0.02dB | Hurt |
| Replace 18 convolution units with real-imaginary Fourier shifting (ablation) | DnCNN · SIDD | PSNR: 37.1992 -> 38.2810 | Helped |
| Replace 18 convolution units with amplitude-phase Fourier shifting (ablation) | DnCNN · SIDD | PSNR: 37.1992 -> 37.5251 | Helped |
| Replace 18 convolution units with spatial shifting (ablation) | DnCNN · SIDD | PSNR: 37.1992 -> 23.6867 | Hurt |
| Vary shifting displacement for Fourier shifting (ablation) | DnCNN · SIDD | PSNR: 37.2309, 37.2663, 37.4277, 37.3781; Fcycle-AP: 37.3781, 37.3551, 37.3139, 37.4754 | Mixed |
| Vary shifting displacement for spatial shifting (ablation) | DnCNN · SIDD | PSNR: 37.2247, 37.3379, 37.4009, 37.1247 | Mixed |
Surprising findings
- The spatial shift operator degraded performance on several restoration baselines, and at large shift displacements it sharply collapsed performance (DnCNN at ns=18 dropped to 23.69 dB PSNR).
Limitations
- Authors acknowledge limited experiments and representative baselines across broader computer vision tasks.
- The amplitude-phase variant involves trigonometric computations that may cause numerical instability in engineering use.
- Positioned as an initial exploration of shifting mechanisms for low-level restoration.
Experimental setup
Datasets: LOL Huawei GoPro HIDE SIDD DND WorldView-II GaoFen2 WorldView-III
Benchmarks: LOL Huawei SIDD DND GoPro HIDE WorldView-II GaoFen2 WorldView-III
Metrics: PSNR SSIM SAM ERGAS mutual information
Baselines: SID DRBN DnCNN DeepDeblur PanNet MutNet INNformer SFINet Spatial shift (Shift-sa)
Builds on: spatial shift operator S2-MLP ShiftViT Deep Fourier Up-Sampling
Closest papers by meaning
- Omni-Kernel Network for Image Restoration (AAAI 2024 · similarity 0.85)
- CosAE: Learnable Fourier Series for Image Restoration (NeurIPS 2024 · similarity 0.85)
- Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration (CVPR 2024 · similarity 0.85)
- Efficient RAW Image Deblurring with Adaptive Frequency Modulation (NeurIPS 2025 · similarity 0.85)
- IR-CM: The Fast and General-purpose Image Restoration Method Based on Consistency Model (NeurIPS 2024 · similarity 0.84)
- TinyLUT: Tiny Look-Up Table for Efficient Image Restoration at the Edge (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.