Research map / Image generation / Image restoration

Parameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts

NeurIPS 2024 · Image generation › Image restoration · approach: Efficient restoration architectures · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdaptIR, a heterogeneous mixture-of-experts parameter-efficient adapter with orthogonal local-spatial, global-spatial (frequency), and channel branches, to adapt frozen pre-trained transformer restoration models to single and hybrid degradations.

Problem

Existing parameter-efficient transfer learning (PETL) methods produce homogeneous representations across restoration tasks, so they perform unstably on single-degradation tasks and poorly on hybrid degradations that need heterogeneous representations.

Approach

A multi-branch MoE adapter whose branches are designed to be orthogonal (depthwise low-rank local convolution, FFT-based frequency affine global module, and spatial-attention channel gating), combined by a degradation-specific adaptive feature ensemble, inserted in parallel to the frozen MLP of a pre-trained restoration transformer.

Key results

• On LR4&Noise30 hybrid degradation (IPT base), AdaptIR reaches 26.48 dB PSNR on Set5 and 22.88 dB on Urban100 with 697K trainable parameters, outperforming FacT (537K) and the classic MoE baseline.
• On the light-deraining setting with all-in-one comparison, AdaptIR (697K params, about 8h fine-tuning on one GPU) reports 37.81 dB PSNR on Rain100L versus 37.04 dB for PromptIR (97M params).
• On LOLv1 low-light enhancement, AdaptIR achieves 19.46 dB PSNR and 0.7441 SSIM, the best among compared PETL methods.

Measured effects reported in the paper

Surprising findings

• The classic MoE baseline with identical branches outperforms existing state-of-the-art PETL methods on hybrid degradation tasks.
• Existing PETL methods such as Adapter and FacT perform well on some single-degradation tasks but drop sharply on others (e.g., low-light enhancement), showing unstable generalization.

Limitations

• All tasks share the same adapter structure, though different restoration tasks may prefer different model designs.
• Evaluated only on a limited set of degradations; blur and haze are not covered.
• Comparison to all-in-one methods uses different training regimes and hardware assumptions, so efficiency comparisons are approximate.

Experimental setup

Datasets: DIV2K Flickr2K BSD400 WED RainTrainL RainTrainH LOLv1

Benchmarks: Set5 Set14 BSDS100 Urban100 Manga109 CBSD68 Rain100L Rain100H LOLv1

Metrics: PSNR SSIM trainable parameter count

Baselines: VPT Adapter LoRA AdaptFormer SSF FacT MoE Full-ft AirNet PromptIR

Builds on: Adapter FacT AdaptFormer IPT

Closest papers by meaning

• Debiased All-in-one Image Restoration with Task Uncertainty Regularization (AAAI 2025 · similarity 0.88)
• Boosting Image Restoration via Priors from Pre-trained Models (CVPR 2024 · similarity 0.88)
• Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration (NeurIPS 2026 · similarity 0.88)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration (ICLR 2025 · similarity 0.87)
• Test-Time Degradation Adaptation for Open-Set Image Restoration (ICML 2024 · similarity 0.87)
• Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restoration (CVPR 2025 · similarity 0.87)