Research map / Image generation / Image restoration
Improving Image Restoration through Removing Degradations in Textual Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Restores images by first removing degradation from CLIP-derived textual embeddings, then using a Stable Diffusion guidance image to inject clean content into existing restoration networks.
Problem
Image restoration struggles because degradations are tightly coupled with image content, making removal hard, especially for all-in-one restoration across multiple degradation types.
Approach
Map degraded images into CLIP textual word embeddings, remove degradation-related concepts in that textual space, generate a clean guidance image with a pretrained text-to-image model, and inject multi-scale guidance features via coarse-to-fine matching into restoration backbones.
Key results
- All-in-one restoration average PSNR 32.56 vs PromptIR 32.06 (+0.5 dB), with +1.05 dB on dehazing SOTS and +1.21 dB on deraining Rain100L.
- Motion deblurring on GoPro: Ours-NAFNet 33.97 dB vs NAFNet 33.69 dB; Ours-Restormer 33.11 dB vs Restormer 32.92 dB.
- Real-world denoising on SIDD: 40.09 dB PSNR vs Restormer 40.02 dB (+0.07 dB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add textual degradation removal and generated guidance | PromptIR · All-in-one restoration average | PSNR: 32.06 -> 32.56 | Helped |
| add textual degradation removal and generated guidance | Restormer · GoPro motion deblurring | PSNR: +0.28 dB PSNR | Helped |
| add textual degradation removal and generated guidance | RestormerS · DPDD single-image defocus deblurring, combined | PSNR: +0.17 dB PSNR | Helped |
| add textual degradation removal and generated guidance | RestormerD · DPDD dual-pixel defocus deblurring, combined | PSNR: +0.16 dB PSNR | Helped |
| add textual degradation removal and generated guidance | DRSformer · Rain200H deraining | PSNR: 32.16 -> 31.97 | Hurt |
| add textual degradation removal and generated guidance | Restormer · Grayscale Gaussian denoising, σ=15 | PSNR | No clear effect |
| set textual representation word count N to 5 (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 31.13 | Helped |
| set textual representation word count N to 10 (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 31.36 | Helped |
| set textual representation word count N to 20 (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 31.57 | Helped |
| set textual representation word count N to 30 (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 31.51 | Helped |
| set textual representation word count N to 40 (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 31.60 | Helped |
| integrate guidance into decoder only (ablation) | NAFNet · GoPro motion deblurring | PSNR: 30.16 -> 30.31 | No clear effect |
Surprising findings
- Using only the I2T mapper or degraded inputs as guidance fails to improve over the baseline, showing the textual degradation removal step is essential.
Limitations
- Gains on some benchmarks are marginal (e.g., +0.07 dB on SIDD, comparable on Rain200H and low-noise denoising).
- Performance on severe rain streaks can be limited because they interfere with the dynamic aggregation process.
Experimental setup
Datasets: BSD400 WED Rain100L SOTS GoPro HIDE RealBlur DPDD RESCAN-Rain200L Rain200H DID-Data DDN-Data
Benchmarks: SOTS Rain100L BSD68 GoPro HIDE RealBlur DPDD Rain200L Rain200H DID-Data DDN-Data Set12
Metrics: PSNR SSIM
Baselines: PromptIR AirNet Restormer NAFNet DRSformer SFNet MPRNet SwinIR
Builds on: Stable Diffusion CLIP PromptIR Restormer NAFNet
Closest papers by meaning
- Adapting Text-to-Image Generation with Feature Difference Instruction for Generic Image Restoration (CVPR 2025 · similarity 0.90)
- Controlling Vision-Language Models for Multi-Task Image Restoration (ICLR 2024 · similarity 0.87)
- Visual-Instructed Degradation Diffusion for All-in-One Image Restoration (CVPR 2025 · similarity 0.86)
- Text-guided Explorable Image Super-resolution (CVPR 2024 · similarity 0.86)
- Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restoration (CVPR 2025 · similarity 0.86)
- Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.