Research map / Image generation / Image restoration
Misalignment-Robust Frequency Distribution Loss for Image Transformation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Frequency Distribution Loss (FDL), which computes sliced Wasserstein distances between DFT amplitude and phase of VGG19 features, as a misalignment-robust training loss for image enhancement, super-resolution, and style transfer.
Problem
Image transformation models (enhancement, super-resolution, style transfer) are usually trained with pixel-aligned pairs, but real paired data is often misaligned; element-wise losses are brittle to misalignment, while spatial-domain distribution losses ignore structure.
Approach
Compute distribution distance (sliced Wasserstein) on the amplitude and phase of DFT-transformed pretrained feature maps, rather than in the spatial domain, so the loss is shift-robust while capturing global structure and multiple image characteristics.
Key results
- On DPED with NAFNet, FDL reaches 23.048 PSNR and 37.501 FID versus 22.256 PSNR and 74.553 FID for CTX.
- On synthetic shifted DIV2K with NLSN on Urban100, FDL reaches 26.702 PSNR versus 25.138 for CTX.
- On merged real-world SISR with SwinIR, FDL reaches 35.771 PSNR versus 35.249 for CTX and 34.733 for PDL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace PDL with FDL | NAFNet · DPED | PSNR: 22.665 -> 23.048 | Helped |
| replace CTX with FDL | SwinIR · DPED | FID: 64.093 -> 29.877 | Helped |
| replace CTX with FDL | NAFNet · RealSR + City100 | PSNR: 24.615 -> 24.865 | Helped |
| replace PDL with FDL | SwinIR · RealSR + City100 | LPIPS: 0.086 -> 0.085 | Helped |
| replace CTX with FDL | NLSN · DIV2K shifted Urban100 | PSNR: 25.138 -> 26.702 | Helped |
| calculate distribution distance in the frequency domain instead of feature space (ablation) | FDL on NAFNet · DPED | PSNR: 22.916 -> 23.048 | Helped |
| use VGG19 rather than ResNet as the feature extractor (ablation) | FDL on NAFNet · DPED | PSNR: 22.415 -> 23.048 | Helped |
| use VGG19 rather than EffNet as the feature extractor (ablation) | FDL on NAFNet · DPED | PSNR: 20.389 -> 23.048 | Helped |
| add a feature extractor instead of calculating FDL directly on pixels (ablation) | FDL on NAFNet · DPED | PSNR: 22.581 -> 23.048 | Helped |
| set λ to 100 rather than 0.01 (ablation) | FDL on NAFNet · DPED | PSNR: 23.048 -> 22.134 | Hurt |
| set λ to 10 rather than 0.01 (ablation) | FDL on NAFNet · DPED | PSNR: 23.048 -> 22.815 | Hurt |
| set λ to 1 rather than 0.01 (ablation) | FDL on NAFNet · DPED | PSNR: 23.048 -> 22.810 | Hurt |
Surprising findings
- Element-wise LPIPS loss performs far worse than distribution-based losses under misalignment, with PSNR dropping to about 16.97 on the NAFNet SISR setting.
- Removing the feature extractor (computing the FDL on raw pixels) gives PSNR 22.581, close to the VGG-based result for enhancement, though VGG19 is still best overall.
Limitations
- Lambda weighting between amplitude and phase must be tuned per task (0.01 for enhancement, 1 for SISR and style transfer).
- Authors note that equal treatment of frequency regions is a limitation and suggest attention weighting across frequency bands as future work.
- Evaluated on a limited set of CNN and Transformer baselines and a single feature extractor family (VGG19).
Experimental setup
Datasets: DPED RealSR City100 DIV2K
Benchmarks: DPED merged RealSR+City100 synthetic shifted DIV2K (Set5, Set14, B100, Urban100)
Metrics: PSNR SSIM LPIPS DISTS FID
Baselines: CTX PDL LPIPS Gatys et al. perceptual loss
Builds on: PDL Contextual Loss Elnekave sliced Wasserstein patch distribution loss Gatys style transfer
Closest papers by meaning
- Efficient RAW Image Deblurring with Adaptive Frequency Modulation (NeurIPS 2025 · similarity 0.85)
- FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing (CVPR 2025 · similarity 0.84)
- Beyond Spatial Domain: Cross-domain Promoted Fourier Convolution Helps Single Image Dehazing (AAAI 2025 · similarity 0.84)
- Improving the Learning Capability of Small-size Image Restoration Network by Deep Fourier Shifting (NeurIPS 2024 · similarity 0.84)
- Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification (CVPR 2024 · similarity 0.84)
- Balancing Frequencies and Pixels in Flow Matching (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.