Research map / Image generation / Image restoration
The Perception-Robustness Tradeoff in Deterministic Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that deterministic estimators with high joint perceptual quality (small Wasserstein distance between joint distributions of estimate and measurement) must have large Lipschitz constants, and validates this on GAN-based super-resolution models.
Problem
Deterministic image restoration methods are often trained to achieve both high perceptual quality and consistency with measurements, but it is unclear what this implies about their stability. The paper asks whether such estimators must be erratic (high Lipschitz constant) under non-invertible degrad
Approach
Shows that the joint perceptual index (Wasserstein distance between p_{X,Y} and p_{X_hat,Y}) lower-bounds the Lipschitz constant of any deterministic estimator as Lip >= m1/sqrt(W) - m2 for non-invertible degradations, independent of degradation type, architecture, or data distribution.
Key results
- Theorem: for non-invertible degradations, Lip(X_hat) >= m1/sqrt(W_p) - m2, so Lipschitz constant grows to infinity as the joint perceptual index approaches zero.
- On DIV2K super-resolution across many GAN-based methods, lower Joint-FID corresponds to larger average Lipschitz lower bound K-bar under I-FGSM with alpha=1/255.
- On CelebA-HQ gender attack with GFPGAN, the female-classified fraction rises from 63.3% (original outputs) to 72.2% (attacked), while the MSE-trained RRDB stays robust (59.1% to 56.9%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reduce the robustness-loss coefficient λ in the CGAN denoiser objective (ablation) | Toy-example deterministic CGAN denoiser · Gaussian denoising toy example | average Lipschitz lower bound K̄ | Mixed |
| Reduce the joint perceptual index | Deterministic image super-resolution algorithms · DIV2K test set with Track1, Track2, and 4× bicubic degradations | average Lipschitz lower bound K̄ | Hurt |
| Perturb the degraded input with an attack targeting the female class | GFPGAN v1.3 · CelebA-HQ gender classification | female-classified images: 63.3% -> 72.2% | Helped |
| Perturb the degraded input with an attack targeting the female class | RRDB · CelebA-HQ gender classification | female-classified images: 59.1% -> 56.9% | Hurt |
| Use an L1-trained RRDB model instead of GFPGAN for the targeted attack | Face-restoration decision pipeline · CelebA-HQ gender classification | vulnerability to targeted attack | Hurt |
Surprising findings
- The MSE-trained RRDB's female-classified fraction decreases (59.1% to 56.9%) even though the attack targets 'female', showing the attack is not uniformly effective for low-perceptual-quality models.
Limitations
- Evaluation is restricted to image super-resolution, although the theorem applies to any non-invertible degradation.
- Evaluated deterministic methods with high joint perceptual quality are all GAN-based.
- Posterior exploration is shown only qualitatively, without quantitative evaluation.
Experimental setup
Datasets: DIV2K CelebA-HQ
Benchmarks: DIV2K test set with Track1 and Track2 degradations and 4x bicubic
Metrics: Lipschitz constant lower bound (average K via I-FGSM) Joint-FID Joint-EMD RMSE FID KID precision and recall
Baselines: RealESRGAN BSRGAN BSRNet FeMaSR DAN ESRGAN RRDB SwinIR-GAN
Builds on: conditional GAN I-FGSM
Closest papers by meaning
- Universal Robustness via Median Randomized Smoothing for Real-World Super-Resolution (CVPR 2024 · similarity 0.87)
- A Restoration Network as an Implicit Prior (ICLR 2024 · similarity 0.87)
- FiRe: Fixed-points of Restoration Priors for Solving Inverse Problems (CVPR 2025 · similarity 0.86)
- A Diffusion Model with State Estimation for Degradation-Blind Inverse Imaging (AAAI 2024 · similarity 0.86)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.86)
- Exact Posterior Score Estimation for Solving Linear Inverse Problems (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.