Research map / Image generation / Image restoration
Robust Image Denoising through Adversarial Frequency Mixup
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adversarial Frequency Mixup (AFM), a model-agnostic training framework that mixes noisy and denoised images in the frequency domain with an adversarially learned mask to generate hard, realistic noise for improving out-of-distribution real-noise denoising.
Problem
Real-world image denoisers trained on a limited set of real noise distributions (SIDD) overfit and generalize poorly to out-of-distribution real noise from other cameras and ISP pipelines.
Approach
Generate augmented training images by adversarially mixing the noisy input and its denoised prediction in the frequency domain via a learned mask (element-wise AFM-E or band-wise AFM-B), so the new noise stays on the real-noise manifold yet is hard for the denoiser.
Key results
- DnCNN with AFM-B improves average OOD PSNR from 37.81 dB to 38.56 dB over normal training on five real-noise benchmarks.
- DnCNN with AFM-B outperforms officially released Restormer by at least +0.39 dB average OOD PSNR (38.56 vs 38.17) with fewer parameters and MACs.
- Naive PGD adversarial training reduces OOD average PSNR by 0.63 dB, while random frequency mixup gives +0.36 dB vs +0.75 dB for AFM-B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train DnCNN with AFM-B | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 38.56 | Helped |
| Train CBDNet with AFM-B | CBDNet · OOD Avg. | PSNR (dB): 38.25 -> 38.58 | Helped |
| Train MPRNet† with AFM-B | MPRNet† · OOD Avg. | PSNR (dB): 38.02 -> 38.56 | Helped |
| Train NAFNet† with AFM-B | NAFNet† · OOD Avg. | PSNR (dB): 37.27 -> 38.27 | Helped |
| Train DnCNN with AFM-B | DnCNN · SIDD validation | PSNR (dB): 38.62 -> 38.35 | Hurt |
| Train DnCNN with random frequency mixup instead of adversarial mask generation (ablation) | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 38.17 | Helped |
| Use adversarial frequency mask generation instead of random frequency mixup (ablation) | DnCNN · OOD Avg. | PSNR (dB): 38.17 -> 38.51 | Helped |
| Use band-wise rather than element-wise adversarial mask generation (ablation) | DnCNN with AFM · OOD Avg. | PSNR (dB): 38.51 -> 38.56 | No clear effect |
| Apply dropout before the last convolution layer (ablation) | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 37.57 | Hurt |
| Randomly mask features after the first convolutional layer (ablation) | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 37.28 | Hurt |
| Apply CutMix data augmentation (ablation) | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 37.96 | Helped |
| Train with PGD adversarial perturbations (ablation) | DnCNN · OOD Avg. | PSNR (dB): 37.81 -> 37.18 | Hurt |
Surprising findings
- Naive PGD adversarial training hurts out-of-distribution robustness (37.18 vs 37.81 dB), suggesting synthetic adversarial noise is harmful without realistic noise properties.
- A small DnCNN trained with AFM beats much larger SOTA supervised denoisers such as Restormer on OOD average PSNR.
Limitations
- Evaluation trained only on SIDD Medium; generalization shown only on five real-noise benchmarks.
- Large architectures (MPRNet, NAFNet) were reduced in size for training cost, so results are not directly comparable to original models.
- Compute resources and training time are not reported.
Experimental setup
Datasets: SIDD Medium
Benchmarks: SIDD validation Poly CC HighISO iPhone Huawei
Metrics: PSNR SSIM
Baselines: DnCNN CBDNet MPRNet NAFNet MIRNet-v2 Uformer Restormer R2R AP-BSN CVF-SID Dropout Input mask
Builds on: mixup CutMix PGD adversarial training NAFNet MPRNet
Closest papers by meaning
- GAN2GAN: Generative Noise Learning for Blind Denoising with Single Noisy Images (ICLR 2021 · similarity 0.88)
- Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training (NeurIPS 2021 · similarity 0.88)
- End-to-End Unpaired Image Denoising with Conditional Adversarial Networks (AAAI 2020 · similarity 0.87)
- (Certified!!) Adversarial Robustness for Free! (ICLR 2023 · similarity 0.87)
- Image Blind Denoising With Generative Adversarial Network Based Noise Modeling (CVPR 2018 · similarity 0.87)
- Is Noise Conditioning Necessary for Denoising Generative Models? (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.