Research map / Image generation / Image restoration
Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Posterior-Mean Rectified Flow (PMRF), which predicts the MMSE posterior mean with a SwinIR regressor and then transports it to the image distribution with a rectified flow, approximating the MSE-optimal perfect-perceptual-index estimator for photo-realistic restoration.
Problem
Photo-realistic image restoration needs low distortion (MSE) without sacrificing perceptual quality; posterior sampling doubles the MMSE and GAN-based methods are hard to optimize, so the minimum-MSE estimator under a perfect perceptual index constraint is not approached well.
Approach
Two-stage method: train a regressor for the posterior mean, then train a rectified flow in image space that transports the posterior-mean prediction (plus small Gaussian noise) to the ground-truth distribution, which theory shows gives MSE no larger than posterior sampling and perfect perceptual index.
Key results
- On CelebA-Test, PMRF achieves the best FID (37.46), KID (0.0257), PSNR (26.37) and SSIM (0.7073) among compared blind face restoration methods.
- PMRF attains the smallest IndRMSE on all real-world sets (LFW, WebPhoto, CelebAdult, WIDER) while staying on par with state-of-the-art FID.
- In controlled experiments PMRF dominates flow-from-Y and posterior-sampling conditional flows in most denoising, super-resolution, inpainting, and colorization tasks on the RMSE-FID plane.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PMRF instead of DifFace | Blind face restoration · CelebA-Test | FID: 38.43 -> 37.46 | Helped |
| Use PMRF instead of DifFace | Blind face restoration · CelebA-Test | KID: 0.0258 -> 0.0257 | Helped |
| Use PMRF instead of VQFRv1 | Blind face restoration · CelebA-Test | NIQE: 3.693 -> 4.118 | Hurt |
| Use PMRF instead of DifFace | Blind face restoration · CelebA-Test | Precision: 0.7413 -> 0.7073 | Hurt |
| Use PMRF instead of BFRffusion | Blind face restoration · CelebA-Test | PSNR: 26.21 -> 26.37 | Helped |
| Use PMRF instead of BFRffusion | Blind face restoration · CelebA-Test | SSIM: 0.6917 -> 0.7073 | Helped |
| Use PMRF instead of CodeFormer | Blind face restoration · CelebA-Test | LPIPS: 0.3432 -> 0.3470 | Hurt |
| Use PMRF instead of RestoreFormer++ | Blind face restoration · CelebA-Test | Deg: 29.63 -> 30.67 | Hurt |
| Use PMRF instead of BFRffusion | Blind face restoration · CelebA-Test | LMD: 1.992 -> 2.030 | Hurt |
| Increase the number of PMRF inference flow steps K (ablation) | PMRF · Image restoration | perceptual quality | Helped |
| Increase the number of PMRF inference flow steps K (ablation) | PMRF · Image restoration | distortion | Hurt |
| Set the source noise level σ_s to 0 (ablation) | PMRF · Image restoration | FID | Hurt |
Surprising findings
- Flow from the degraded measurement Y performs about as well as PMRF on denoising, though the paper proves both are optimal in the Gaussian toy example.
- Conditioning a flow on the posterior mean (rather than Y) gives theoretically the same MSE as posterior sampling, so the gain comes from the unconditional transport formulation.
Limitations
- The noise level sigma_s must be tuned; too large or too small harms MSE or perceptual quality.
- The MSE guarantee relies on assumptions (perfect perceptual index, non-degenerate residual) that may be restrictive.
- Real-world evaluation uses IndRMSE as a proxy since ground truth is unavailable.
Experimental setup
Datasets: FFHQ ImageNet CelebA-HQ LFW-Test WebPhoto-Test CelebAdult-Test WIDER-Test
Benchmarks: CelebA-Test LFW-Test WebPhoto-Test CelebAdult-Test WIDER-Test ImageNet validation (controlled experiments)
Metrics: FID KID NIQE Precision PSNR SSIM LPIPS Deg LMD IndRMSE
Baselines: DOT RestoreFormer++ RestoreFormer CodeFormer VQFRv1 VQFRv2 GFPGAN DiffBIR DifFace BFRffusion flow from Y flow conditioned on Y
Builds on: rectified flow Deep Optimal Transport (DOT) flow matching distortion-perception tradeoff
Closest papers by meaning
- Reversing Flow for Image Restoration (CVPR 2025 · similarity 0.89)
- Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration (ICML 2024 · similarity 0.87)
- PnP-Flow: Plug-and-Play Image Restoration with Flow Matching (ICLR 2025 · similarity 0.87)
- Exact Posterior Score Estimation for Solving Linear Inverse Problems (NeurIPS 2026 · similarity 0.87)
- SP$^3$: Spherical Priors for Plug-and-Play Restoration (NeurIPS 2026 · similarity 0.87)
- Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.