Research map / Image generation / Image restoration

EnsIR: An Ensemble Algorithm for Image Restoration via Gaussian Mixture Models

NeurIPS 2024 · Image generation › Image restoration · approach: Efficient restoration architectures · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Reformulates post-training ensemble of image restoration models as range-wise univariate Gaussian mixtures solved by EM, storing weights in a lookup table for training-free inference.

Problem

Post-training ensembling of pre-trained image restoration models is hard because traditional regression-based ensemble methods are inefficient and averaging ignores performance variation across pixel value ranges and models.

Approach

Model pixel predictions as Gaussian mixtures over histogram-like value bins, estimate per-bin ensemble weights with a mean-prior EM algorithm on a small reference set, and store them in a lookup table for fast, training-free inference on test images.

Key results

• On 14 benchmarks across super-resolution, deblurring and deraining, EnsIR outperforms regression-based and averaging ensembles; e.g., Rain100L PSNR 39.216 vs 39.115 (HGBT) and 38.675 (Average).
• On Test100 with MPRNet as a weak base model, EnsIR reaches 32.064 PSNR versus 31.626 for Average.
• With bin width 64 the method reaches real-time inference at a small performance cost, while being much faster than regression-based methods (e.g., 0.1709 s vs ~9.86 s for RForest on Rain100H with b=32).

Measured effects reported in the paper

Surprising findings

• Averaging and the ZZPM weighting often perform no better than averaging, and regression-based ensembles can outperform them when one base model is much weaker.

Limitations

• Slower than simple averaging and ZZPM; real-time requires sacrificing performance.
• If all base models fail, the ensemble cannot produce a better result.
• Relies on the reference set being drawn from the same distribution as the test set.

Experimental setup

Datasets: DIV2K GoPro Rain13K

Benchmarks: Set5 Set14 BSDS100 Urban100 Manga109 GoPro HIDE RealBlur-R RealBlur-J Rain100H Rain100L Test100

Metrics: PSNR SSIM runtime per image

Baselines: Averaging Bagging AdaBoost Random forests GBDT HGBT ZZPM RefESR

Builds on: RefESR bagging boosting histogram gradient boosting

Closest papers by meaning

• Parameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts (NeurIPS 2024 · similarity 0.87)
• Complexity Experts are Task-Discriminative Learners for Any Image Restoration (CVPR 2025 · similarity 0.86)
• Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration (NeurIPS 2026 · similarity 0.86)
• Stochastic Deep Restoration Priors for Imaging Inverse Problems (ICML 2025 · similarity 0.85)
• RestoreGrad: Signal Restoration Using Conditional Denoising Diffusion Models with Jointly Learned Prior (ICML 2025 · similarity 0.85)
• FiRe: Fixed-points of Restoration Priors for Solving Inverse Problems (CVPR 2025 · similarity 0.85)