Research map / Image generation / Image restoration
Degradation-Aware Feature Perturbation for All-in-One Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DFPIR, a Restormer-based all-in-one image restorer that inserts degradation-guided channel shuffling and top-K attention masking (DGPB) between encoder and decoder, guided by CLIP degradation prompts.
Problem
Task interference in all-in-one image restoration: shared parameters across diverse degradations (noise, haze, rain, blur, low light) cause conflicting gradient directions and degraded performance.
Approach
Instead of modifying parameters or adding prompt-based feature modulation, perturb encoder features in a degradation-guided way: channel shuffling in a doubled channel space (DGCPM) followed by top-K attention masking in a channel-adapted cross-attention (CAAPM), so features align with shared decoder parameters.
Key results
- On the three-task all-in-one setting, DFPIR averages 32.88 dB PSNR, 0.45 dB above InstructIR (32.43 dB) and 0.82 dB above PromptIR.
- On the five-task all-in-one setting, DFPIR averages 30.64 dB PSNR (0.913 SSIM), 1.09 dB above InstructIR (29.55 dB).
- Ablation: channel shuffle (DGCPM) alone gives 32.49 dB vs 32.34 dB for channel attention; adding CAAPM to DGCPM reaches 32.88 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DFPIR instead of InstructIR | All-in-one restoration model · Three-task average | PSNR (dB): 32.43 -> 32.88 | Helped |
| use DFPIR instead of InstructIR | All-in-one restoration model · Five-task average | PSNR (dB): 29.55 -> 30.64 | Helped |
| add channel attention only (ablation) | Restormer all-in-one model · Three-task average | PSNR (dB): 31.98 -> 32.34 | Helped |
| replace channel attention with DGCPM channel shuffling (ablation) | Restormer all-in-one model · Three-task average | PSNR (dB): 32.34 -> 32.49 | Helped |
| add CAAPM to channel attention (ablation) | Restormer all-in-one model · Three-task average | PSNR (dB): 32.34 -> 32.65 | Helped |
| add CAAPM attention perturbation to DGCPM (ablation) | Restormer all-in-one model · Three-task average | PSNR (dB): 32.49 -> 32.88 | Helped |
| set perturbation factor γ to 0.5 rather than 0.9 (ablation) | DFPIR · Three-task average | PSNR (dB): 32.88 -> 32.67 | Hurt |
| set perturbation factor γ to 0.7 rather than 0.9 (ablation) | DFPIR · Three-task average | PSNR (dB): 32.88 -> 32.81 | Hurt |
| set perturbation factor γ to 0.8 rather than 0.9 (ablation) | DFPIR · Three-task average | PSNR (dB): 32.88 -> 32.84 | Hurt |
| set perturbation factor γ to 1.0, removing attention perturbation (ablation) | DFPIR · Three-task average | PSNR (dB): 32.88 -> 32.83 | Hurt |
Surprising findings
- Channel shuffling in a low-dimensional space was found to make training hard to converge, so the features are expanded 2x before shuffling.
Limitations
- The paper states that the perturbation factor gamma must trade off inter-task interference against information loss, requiring tuning (fixed at 0.9).
- Evaluation is limited to standard synthetic benchmarks with PSNR/SSIM only.
Experimental setup
Datasets: BSD400 WED SOTS Rain100L GoPro LOL-v1 CBSD68 Urban100
Benchmarks: SOTS Rain100L CBSD68 Urban100 GoPro LOL-v1
Metrics: PSNR SSIM
Baselines: PromptIR InstructIR AirNet IDR Restormer NAFNet DGUNet MPRNet TransWeather DL
Builds on: Restormer PromptIR MedIR routing strategy CLIP
Closest papers by meaning
- DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor (NeurIPS 2024 · similarity 0.88)
- AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation (ICLR 2025 · similarity 0.87)
- Learning Where and What to Restore for Composite Image Restoration (NeurIPS 2026 · similarity 0.87)
- Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration (NeurIPS 2026 · similarity 0.87)
- Visual-Instructed Degradation Diffusion for All-in-One Image Restoration (CVPR 2025 · similarity 0.87)
- Debiased All-in-one Image Restoration with Task Uncertainty Regularization (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.