Research map / Image generation / Image restoration
Towards Progressive Multi-Frequency Representation for Image Warping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MFR, a progressive multi-frequency filtering network with learnable Gabor wavelet filters that learns frequency representations for image warping, evaluated on homography, ERP-to-perspective, and asymmetric SR.
Problem
Interpolation-based and learned-resampling-kernel image warping methods produce distorted images with weak high-frequency detail, and generalize poorly to out-of-distribution scaling factors and transformations.
Approach
A progressive filtering network that stacks frequency learning modules (sinusoidal/Gabor filters with gating and short connections) to add fine-scale details coarse-to-fine on top of a bicubic-upsampled image, with learnable 1-D Gabor wavelet filters to capture local spatial-frequency variations.
Key results
- In the out-of-scale homography setting, MFR-RRDB reaches 27.12 mPSNR on DIV2KW, 0.20 dB above LTEW-RRDB (26.92).
- On Urban100W out-of-scale homography, MFR-RRDB achieves 24.51 mPSNR, 0.26 dB above LTEW-RRDB.
- For asymmetric SR, MFR-RCAN is best or second-best across most in-scale and out-of-scale settings, with only small PSNR gains over LTEW-RCAN (e.g., 29.65 vs 29.62 on B100 in-scale).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MFR-RRDB instead of LTEW-RRDB | RRDB-based homography warping · DIV2KW homography (out-of-scale) | mPSNR: 26.92 -> 27.12 | Helped |
| Use MFR-RRDB instead of LTEW-RRDB | RRDB-based homography warping · Urban100W homography (out-of-scale) | mPSNR: 24.25 -> 24.51 | Helped |
| Use MFR-RCAN instead of LTEW-RCAN | RCAN asymmetric super-resolution · Set5 asymmetric SR (out-of-scale, ×3.0) | PSNR(dB): 29.26 -> 29.27 | Helped |
| Use learnable Gabor wavelet filters instead of SIREN filters (ablation) | MFR · DIV2KW homography (out-of-scale) | mPSNR: 26.81 -> 26.90 | Helped |
| Use learnable Gabor wavelet filters instead of static Gabor wavelet filters (ablation) | MFR · DIV2KW homography (out-of-scale) | mPSNR: 26.86 -> 26.90 | Helped |
| Use learnable Gabor wavelet filters instead of conventional Gabor filters (ablation) | MFR · DIV2KW homography (out-of-scale) | mPSNR: 26.86 -> 26.90 | Helped |
| Add a short connection (ablation) | MFR · DIV2KW homography (in-scale) | mPSNR: 30.79 -> 30.88 | Helped |
| Add the gate mechanism (ablation) | MFR · DIV2KW homography (in-scale) | mPSNR: 30.72 -> 30.88 | Helped |
| Add a short connection and gate mechanism (ablation) | MFR · Urban100W homography (out-of-scale) | mPSNR: 24.07 -> 24.51 | Helped |
Surprising findings
- Filter choice has marginal impact in the in-scale setting but a substantial impact in the out-of-scale setting.
- Without the short connection, the frequency-representation-only model performs worse than with it (e.g., 30.72 vs 30.88 mPSNR on DIV2KW in-scale).
Limitations
- Optimal network structure may need to be tailored to specific applications and scenarios, per the authors.
- Gains over LTEW in asymmetric SR are small in many settings.
Experimental setup
Datasets: DIV2K
Benchmarks: DIV2KW Set5W Set14W B100W Urban100W Set5 Set14 B100 Urban100 Flickr360
Metrics: mPSNR PSNR
Baselines: Bicubic RRDB SRWarp LTEW MetaSR ArbSR
Builds on: LTEW (local texture estimator warping) SRWarp SIREN RRDB RCAN
Closest papers by meaning
- Neural Implicit Morphing of Face Images (CVPR 2024 · similarity 0.83)
- Misalignment-Robust Frequency Distribution Loss for Image Transformation (CVPR 2024 · similarity 0.83)
- Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks (AAAI 2024 · similarity 0.82)
- DiMSUM: Diffusion Mamba - A Scalable and Unified Spatial-Frequency Method for Image Generation (NeurIPS 2024 · similarity 0.82)
- Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation (AAAI 2025 · similarity 0.82)
- Frequency-Guided Masking for Enhanced Vision Self-Supervised Learning (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.