Research map / Image generation / Super-resolution
Exploring the Low-Pass Filtering Behavior in Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reports a "sinc phenomenon" in ISR networks (impulse responses resemble sinc low-pass filters), proposes Hybrid Response Analysis (HyRA) to split a network into a linear low-pass part and a nonlinear high-frequency-injecting part, and introduces the FSDS frequency-spectrum metric.
Problem
Deep super-resolution networks are treated as black boxes, and existing metrics (PSNR, SSIM, LPIPS) do not assess image quality from a frequency-spectrum perspective, so high-frequency distortions can go unnoticed.
Approach
Probe ISR networks with a Dirac impulse to reveal sinc-shaped responses, decompose the network into a linear (LTI, low-pass) system defined by its impulse response plus a nonlinear system with zero impulse response (HyRA), and quantify injected high frequencies with a spectrum-based similarity metric (FSDS).
Key results
- The paper reports that the FSDS metric (proposed) and existing metrics disagree on rankings, e.g., at x2 SwinIR-classical scores best on FSDS (40.372) while GRLBase scores best on PSNR (35.66) and SSIM (0.945) on DIV2K.
- An x2 low-pass filter sweep finds PSNR peaks at 31.40 (omega=48) and SSIM at 0.87 (omega=45.8), used as reference thresholds.
- Impulse responses of most CNN and transformer ISR networks resemble windowed 2D sinc functions, while SwinIR-Real and ESRGAN (adversarially trained) do not.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Set low-pass filter cutoff to ω=48 | Low-pass filtering for ×2 super-resolution · DIV2K val ×2 | PSNR: 31.40 | Helped |
| Set low-pass filter cutoff to ω=45.8 | Low-pass filtering for ×2 super-resolution · DIV2K val ×2 | SSIM: 0.87 | Helped |
| Use an impulse response more similar to a sinc function | ISR networks · Image super-resolution | Performance | Helped |
| Apply adversarial loss rather than ℓ1 or ℓ2 loss | SwinIR-Real and ESRGAN · Impulse response | Sinc-like response | Hurt |
| Embed content using frequency-spectrum steganography | Image evaluated by FSDS and SSIM · Frequency-spectrum steganography example | FSDS | Hurt |
| Embed content using frequency-spectrum steganography | Image evaluated by FSDS and SSIM · Frequency-spectrum steganography example | SSIM: 0.995 | No clear effect |
| Apply JPEG compression at quality 10 | Image evaluated by FSDS and SSIM · JPEG compression example | FSDS: 26.39dB | Hurt |
| Apply JPEG compression at quality 10 | Image evaluated by FSDS and SSIM · JPEG compression example | SSIM: 0.842 | Hurt |
Surprising findings
- Impulse responses of ISR networks resemble sinc functions, and networks with better performance tend to have responses closer to sinc.
- In a steganography example, FSDS drops to 26.37 dB while SSIM remains at 0.995, showing metrics can disagree on high-frequency distortion.
Limitations
- Sinc-like impulse responses are not present in all networks (e.g., adversarially trained SwinIR-Real and ESRGAN), so the phenomenon is not universal.
- HyRA relies on approximate spatial invariance, which is proven only for convolutional layers and checked experimentally for transformers.
- Analysis is restricted to x2 to x4 (and some x6/x12) ISR on DIV2K validation, with a 4x factor chosen for impulse response visualization.
Experimental setup
Datasets: DIV2K
Benchmarks: DIV2K
Metrics: PSNR SSIM LPIPS FSDS
Baselines: PSNR SSIM LPIPS EDSR RDN SwinIR HAT GRL
Builds on: LAM LIIF Frequency-Principle
Closest papers by meaning
- Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder (CVPR 2024 · similarity 0.86)
- Noise-Free Optimization in Early Training Steps for Image Super-resolution (AAAI 2024 · similarity 0.86)
- ResDiff: Combining CNN and Diffusion Model for Image Super-resolution (AAAI 2024 · similarity 0.86)
- HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution (CVPR 2025 · similarity 0.86)
- Hierarchical Neural Operator Transformer with Learnable Frequency-aware Loss Prior for Arbitrary-scale Super-resolution (ICML 2024 · similarity 0.86)
- SAVSR: Arbitrary-Scale Video Super-Resolution via a Learned Scale-Adaptive Network (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.