Research map / Image generation / Image restoration

Boosting Image Restoration via Priors from Pre-trained Models

CVPR 2024 · Image generation › Image restoration · approach: Priors for restoration networks · 19 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PTG-RM, a lightweight (<1M params) plug-in refinement module that uses off-the-shelf features from frozen pre-trained models (CLIP, BLIP2, Stable Diffusion, restoration models) to refine outputs of existing image restoration networks across low-light enhancement, deraining, deblurring, and denoising.

Problem

Existing image restoration networks have hit performance upper bounds, and strong priors are hard to estimate; pre-trained large-scale models contain degradation-related information that has been unexplored for low-level restoration.

Approach

Use features from a frozen pre-trained model as a prior to learn spatially-varying operation ranges (mixing short-range CNN and long-range transformer features via learned spatial weights) and pre-trained-guided channel-spatial attention with spatially varying kernels, trained jointly with the restoration network and not needed at inference.

Key results

• On LOL-real, SNR-Net improves from 21.48 to 25.50 dB PSNR with the CLIP-prior refinement module (PTG-RM-c), and URetinex from 21.16 to 24.70 dB.
• On SIDD, Restormer with the refinement module reaches 40.22 dB PSNR versus 39.77 dB... (Restormer 40.02 to 40.22 dB, surpassing 40.2 dB).
• The refinement module adds under 1M parameters and improves all tested restoration networks across low-light enhancement, deraining, deblurring, defocus deblurring, denoising, and an unsupervised low-light setting, with a user study of 80 participants preferring enhanced results.

Measured effects reported in the paper

Surprising findings

• Pre-trained models (CLIP, BLIP2, Stable Diffusion) that were not trained for restoration provide useful degradation-related priors for restoration networks, even when the restoration networks have reached their upper bounds.
• Simply concatenating priors with features gives much smaller gains than the proposed spatially-aware mechanism, and simply enlarging the refinement module does not match the gains.

Limitations

• Improvement magnitude varies across experiments and depends on target network capacity and task difficulty.
• Gains are smaller for some strong target models that are near their upper bounds.
• Relies on a choice of pre-trained model (CLIP chosen for efficiency), with no systematic comparison of which prior works best.

Experimental setup

Datasets: LOL-real SID Rain13K Rain100H Rain100L Test100 Test1200 Test2800 Set12 BSD68 CBSD68 Kodak24

Benchmarks: LOL-real SID Rain100H Rain100L Test100 Test1200 Test2800 GoPro HIDE RealBlur-R RealBlur-J DPDD

Metrics: PSNR SSIM LPIPS

Baselines: UHD URetinex SNR SKF SMG SPAIR Restormer MPRNet Uformer IFAN GRL DRUNet

Builds on: SNR-Net CLIP-IQA Restormer SKF SMG

Closest papers by meaning

• Intra and Inter Parser-Prompted Transformers for Effective Image Restoration (AAAI 2025 · similarity 0.88)
• Parameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts (NeurIPS 2024 · similarity 0.88)
• Controlling Vision-Language Models for Multi-Task Image Restoration (ICLR 2024 · similarity 0.88)
• IRPO: Boosting Image Restoration via Post-training GRPO (NeurIPS 2026 · similarity 0.87)
• Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration (AAAI 2024 · similarity 0.87)
• InstructRestore: Region-Customized Image Restoration with Human Instructions (NeurIPS 2025 · similarity 0.87)