Research map / Image generation / Super-resolution
Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SFD, a GAN-based perceptual super-resolution method that discriminates CLIP semantic features (pixel-wise via Feat-D and output via text-guided learnable prompt pairs), and reuses the trained discriminators for opinion-unaware no-reference IQA (SFD-IQA).
Problem
GAN-based super-resolution discriminators operate coarsely on raw images and ignore semantic information, so SR networks struggle to generate fine-grained, semantically consistent textures; existing discriminator-based opinion-unaware NR-IQA also lacks semantic awareness.
Approach
Perform adversarial discrimination on frozen CLIP image-encoder features rather than on pixels: a multi-scale feature discriminator (Feat-D) on middle features, plus a text-guided discriminator (TG-D) using learnable prompt pairs on the final CLIP output; the trained Feat-D and prompt pairs are then reused directly as an OU NR-IQA scorer.
Key results
- On ×4 classical SR (Set5), the SFD-based RRDB model reports PSNR 31.63 and LPIPS 0.063, with best PSNR/SSIM among GAN methods on all benchmarks.
- SFD-IQA reaches PLCC 0.4252 on PIPAL and 0.7574 on KonIQ-10k, best across all four OU NR-IQA datasets in the reported table.
- On PIPAL GAN-based SR subset, adding Feat-D and TG-D raises PLCC from 0.2154 to 0.4006 (+86.0%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Feat-D and TG-D to SISR training (ablation) | SFD · DIV2K100 ×4 | PSNR | Helped |
| Add Feat-D and TG-D to SISR training (ablation) | SFD · DIV2K100 ×4 | LPIPS | Helped |
| Remove Feat-D (ablation) | SFD-IQA · PIPAL GAN.SR | PLCC: 0.4006 vs. 0.2430 | Hurt |
| Remove TG-D (ablation) | SFD-IQA · PIPAL GAN.SR | PLCC: 0.4006 vs. 0.3958 | Hurt |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL GAN.SR | PLCC: 0.2154 -> 0.4006 (+86.0%) | Helped |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL GAN.SR | SRCC: 0.2376 -> 0.3061 (+28.8%) | Helped |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL Trad.SR | PLCC: 0.3787 -> 0.3797 (+0.26%) | No clear effect |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL Trad.SR | SRCC: 0.3545 -> 0.3522 (-0.64%) | Hurt |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL PSNR.SR | PLCC: 0.4290 -> 0.4657 (+8.55%) | Helped |
| Add Feat-D and TG-D (ablation) | SFD-IQA · PIPAL PSNR.SR | SRCC: 0.4432 -> 0.4932 (+11.3%) | Helped |
| Add Feat-D and TG-D (ablation) | SFD-IQA · KonIQ-10k test | PLCC: 0.6865 -> 0.7576 (+10.4%) | Helped |
| Add Feat-D and TG-D (ablation) | SFD-IQA · KonIQ-10k test | SRCC: 0.6862 -> 0.7475 (+8.89%) | Helped |
Surprising findings
- Feat-D and TG-D are used directly for IQA without any IQA-specific training, yet outperform dedicated OU NR-IQA baselines.
Limitations
- Real-world SR results show mixed perceptual metrics (e.g., DISTS worse than some baselines on DRealSR), with only competitive rather than best perceptual scores.
- SFD-IQA's score depends on hyperparameters α1, α2 for weighting and on the CLIP-based discriminators learned during SR training.
Experimental setup
Datasets: DF2K OutdoorSceneTraining Set5 Set14 Urban100 DIV2K validation RealSR DRealSR RealD2K PIPAL Ma's dataset KonIQ-10k
Benchmarks: Set5 Set14 Urban100 DIV2K100 RealSR DRealSR RealD2K PIPAL Ma's dataset KonIQ-10k LIVE-itW Rain100L
Metrics: PSNR SSIM LPIPS DISTS NIQE PLCC SRCC KRCC
Baselines: ESRGAN SRGAN SFTGAN SPSR LDL DualFormer WGSR SeD BSRGAN RealESRGAN RealSwinIR CLIPIQA
Builds on: ESRGAN CLIP-IQA RecycleD SeD CLIP
Closest papers by meaning
- SeD: Semantic-Aware Discriminator for Image Super-Resolution (CVPR 2024 · similarity 0.93)
- Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity (ICML 2024 · similarity 0.88)
- Augmenting Perceptual Super-Resolution via Image Quality Predictors (CVPR 2025 · similarity 0.88)
- Auto-Encoded Supervision for Perceptual Image Super-Resolution (CVPR 2025 · similarity 0.87)
- Uncertainty-Aware GAN for Single Image Super Resolution (AAAI 2024 · similarity 0.87)
- Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.