Research map / Image generation / Super-resolution
Augmenting Perceptual Super-Resolution via Image Quality Predictors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses no-reference image quality assessment (MUSIQ) to weight multi-ground-truth sampling and to directly fine-tune super-resolution models with a LoRA-regularized quality loss, matching or beating human-ranked HGGT without human annotations.
Problem
Perceptual super-resolution relies on human rankings of enhanced ground truths (HGGT) and perceptual/adversarial losses; human ranking is coarse, costly, and non-differentiable, so the paper asks whether NR-IQA metrics can replace humans for SR training.
Approach
Analyze NR-IQA metrics against human preferences on SBS180K and HGGT, then use MUSIQ both to softmax-weight/argmax-select among enhanced GTs per patch (AMO) and as a differentiable loss term during LoRA-regularized fine-tuning to avoid adversarial artifacts.
Key results
- SwinIR-AMO+FT reaches MUSIQ 70.81 vs 66.39 for the human-ranked SwinIR-UPos baseline, while LPIPS-ST improves about 6%.
- On Real-ESRGAN, RESRGAN-AMO+FT gives LPIPS-ST improvement of roughly 12% over RESRGAN-UPos and surpasses UPos on every NR metric (MUSIQ 71.67 vs 65.93).
- Among 42 NR-IQA metric variants, MUSIQ had the lowest positive misalignment rate (0.16) on the HGGT subset, leading to its selection for sampling and optimization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add online MUSIQ-based ground-truth sampling and MUSIQ fine-tuning | SwinIR · HGGT Test-100 | MUSIQ: 66.39 -> 70.81 | Helped |
| Add online MUSIQ-based ground-truth sampling and MUSIQ fine-tuning | Real-ESRGAN · HGGT Test-100 | MUSIQ: 65.93 -> 71.67 | Helped |
| Add online MUSIQ-based ground-truth sampling and MUSIQ fine-tuning | SwinIR · HGGT Test-100 | PSNR: 22.30 -> 21.77 | Hurt |
| Add online MUSIQ-based ground-truth sampling and MUSIQ fine-tuning | Real-ESRGAN · HGGT Test-100 | LPIPS-ST: 0.192 -> 0.169 | Helped |
| Add online MUSIQ-based ground-truth sampling without fine-tuning (ablation) | SwinIR · HGGT Test-100 | MUSIQ: 66.39 -> 68.08 | Helped |
| Add softmax sampling from all ground-truth candidates (ablation) | SwinIR · HGGT Test-100 | MUSIQ: 66.39 -> 66.73 | Helped |
| Add softmax sampling from human-ranked positive ground truths (ablation) | SwinIR · HGGT Test-100 | MUSIQ: 66.39 -> 66.83 | Helped |
| Increase the perceptual-loss weight during fine-tuning (ablation) | SwinIR-UPos + FT · HGGT Test-100 | LPIPS: 0.169 -> 0.166 | Helped |
| Increase the perceptual-loss weight during fine-tuning (ablation) | SwinIR-UPos + FT · HGGT Test-100 | MUSIQ: 69.70 -> 68.38 | Hurt |
| Retain GAN loss during fine-tuning (ablation) | SwinIR-UPos + FT · HGGT Test-100 | MUSIQ: 69.70 -> 69.37 | Hurt |
| Add online MUSIQ-based ground-truth sampling without fine-tuning (ablation) | Real-ESRGAN · HGGT Test-100 | MUSIQ: 65.93 -> 67.86 | Helped |
| Add softmax sampling from all ground-truth candidates (ablation) | Real-ESRGAN · HGGT Test-100 | MUSIQ: 65.93 -> 65.87 | No clear effect |
Surprising findings
- Naive fine-tuning against an NR-IQA metric produces structured adversarial-like artifacts, which LoRA regularization removes.
- Human-labeled positives with SMP sampling performed nearly identically to SMA, which uses no human labels.
Limitations
- NR-IQA metrics fail to recognize negative (low-quality) enhanced ground truths, with negative misalignment rates of 0.45 to 0.97.
- Optimizing against a neural NR-IQA model can introduce structured artifacts that fool the metric and degrade low-level fidelity (PSNR/SSIM drop).
- The gains on NR metrics may partly reflect the metric being optimized; the RealESRGAN AMO+FT does not surpass UPos+FT, which uses human labels.
Experimental setup
Datasets: HGGT SBS180K RealSR
Benchmarks: HGGT Test-100 RealSR
Metrics: PSNR SSIM LPIPS LPIPS-ST DISTS MUSIQ NIMA Q-Align TOPIQ
Baselines: HGGT UPos OrigsOnly Rand SMA SMP AMO UPos+FT
Builds on: HGGT Real-ESRGAN SwinIR MUSIQ LoRA
Closest papers by meaning
- Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption (CVPR 2025 · similarity 0.89)
- Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare (NeurIPS 2024 · similarity 0.89)
- Auto-Encoded Supervision for Perceptual Image Super-Resolution (CVPR 2025 · similarity 0.88)
- DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution (NeurIPS 2025 · similarity 0.88)
- Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach (CVPR 2025 · similarity 0.88)
- Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessment (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.