Research map / Image generation / Super-resolution
Text-guided Explorable Image Super-resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes zero-shot text-guided explorable super-resolution by adapting zero-shot diffusion restoration (DPS, ΠGDM, DDNM) to text-to-image models (Imagen, unCLIP) and by adding CLIP guidance to DDNM.
Problem
Extreme image super-resolution at large downsampling factors is severely ill-posed; existing explorable SR methods are restricted to specific classes (e.g., faces) or trained for specific degradations, and cannot offer open-domain, text-controlled exploration of solutions.
Approach
Use pretrained text-to-image diffusion cascades (Imagen/DeepFloyd IF, unCLIP/Karlo) as zero-shot priors for super-resolution by modifying their multi-stage sampling with DPS, ΠGDM, or DDNM consistency, and alternatively inject CLIP text-image similarity gradients into DDNM sampling.
Key results
- On NoCaps 16x SR, Imagen+DDNM reaches CLIP score 0.3102 (8x) and 0.3037 (16x) versus 0.2418 for DPS at 16x, with LR PSNR above 45 dB for all methods.
- CLIP-guided DDNM improves NIQE on NoCaps 16x (5.71 vs 13.24 for DDNM) while raising CLIP score to 0.2517 from 0.2162.
- In a user study, Imagen-DDNM was preferred for text similarity (90.89% faces, 92.88% NoCaps) over unCLIP-DDNM and CLIP-guided DDNM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Imagen + DDNM instead of unconditional DDNM | Zero-shot super-resolution · NoCaps, 8× | NIQE: 10.27 -> 4.62 | Helped |
| Use Imagen + DDNM instead of unconditional DDNM | Zero-shot super-resolution · NoCaps, 8× | CLIP: 0.2326 -> 0.3102 | Helped |
| Add CLIP guidance to DDNM | DDNM · NoCaps, 8× | CLIP: 0.2326 -> 0.2564 | Helped |
| Add CLIP guidance to DDNM | DDNM · NoCaps, 8× | LR PSNR (dB): 72.94 -> 47.86 | Hurt |
| Enable classifier-free guidance with step size 0.5 instead of disabling it with step size 1.0 (ablation) | Imagen + ΠGDM · NoCaps | LR PSNR (dB): 49.67 -> 24.31 | Hurt |
| Enable classifier-free guidance with step size 0.5 instead of disabling it with step size 1.0 (ablation) | Imagen + ΠGDM · NoCaps | CLIP score: 0.2788 -> 0.2923 | Helped |
| Increase reverse diffusion steps from 100 to 200 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | LR PSNR (dB): 30.675 -> 40.025 | Helped |
| Increase reverse diffusion steps from 100 to 200 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | CLIP score: 0.3011 -> 0.2742 | Hurt |
| Increase reverse diffusion steps from 200 to 300 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | LR PSNR (dB): 40.025 -> 42.431 | Helped |
| Increase reverse diffusion steps from 200 to 300 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | CLIP score: 0.2742 -> 0.2773 | Helped |
| Increase reverse diffusion steps from 300 to 500 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | LR PSNR (dB): 42.431 -> 43.093 | Helped |
| Increase reverse diffusion steps from 300 to 500 with classifier-free guidance (ablation) | Imagen + ΠGDM · CelebA-HQ faces | CLIP score: 0.2773 -> 0.2695 | Hurt |
Surprising findings
- Classifier-free guidance conflicts with gradient-based measurement guidance, sharply reducing LR PSNR (e.g., 100-step NoCaps from 49.67 to 24.31 dB) in T2I-ΠGDM.
- Higher LR PSNR does not always yield better text-aligned results; increasing diffusion steps improves consistency but reduces text adherence.
Limitations
- T2I-DDNM can produce unrealistic images because null-space rectification makes any hallucination data consistent.
- Generated results depend on and inherit biases of the pretrained T2I model and often need several attempts.
- Gradient-based reconstruction guidance trades off against text adherence, and CLIP guidance is outperformed by T2I-based methods.
Experimental setup
Datasets: CelebA-HQ NoCaps
Benchmarks: CelebA-HQ NoCaps
Metrics: LR PSNR NIQE CLIP score user preference (text similarity, photo-realism)
Baselines: DPS DDNM ΠGDM Imagen+DDNM unCLIP+DDNM CLIP guided DDNM
Builds on: DPS DDNM ΠGDM unCLIP Imagen
Closest papers by meaning
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (CVPR 2024 · similarity 0.89)
- Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders (NeurIPS 2025 · similarity 0.88)
- TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.88)
- SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning (NeurIPS 2025 · similarity 0.88)
- PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution (NeurIPS 2026 · similarity 0.87)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.