Research map / Image generation / Image restoration

Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models

NeurIPS 2024 · Image generation › Image restoration · approach: Efficient restoration architectures · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretically shows an uncertainty-perception tradeoff in generative image restoration (entropy power of the error vs. conditional Rényi divergence), implying perfect perception costs up to twice the inherent uncertainty, and validates it on super-resolution and inpainting.

Problem

Generative restoration models achieve high perceptual quality but hallucinate realistic details absent from ground truth; the relationship between perceptual quality and the uncertainty of the restoration error is not well understood.

Approach

Defines an uncertainty-perception function using the conditional entropy power of the recovery error and proves it is lower-bounded by the inherent uncertainty and upper-bounded by twice the Gaussian-based inherent uncertainty; with Rényi-1/2 divergence it yields a closed-form tradeoff region and implies the distortion-perception tradeoff via MSE.

Key results

• Theory: the minimal uncertainty U(P) satisfies N(X|Y) <= U(P) <= 2N(X_G|Y), so perfect perceptual quality entails at least twice the inherent uncertainty for Gaussian cases.
• With Rényi-1/2 divergence, the uncertainty-perception region is eta(P)·N(X|Y) <= U(P) <= eta(P)·N(X_G|Y) with 1 <= eta(P) <= 2, yielding an impossible region in the uncertainty-perception plane.
• Experiments on 4x SR (BSD100) and inpainting (SeeTrue) with several generative models show an empirical anti-correlation between perception and uncertainty, and uncertainty correlates with distortion.

Measured effects reported in the paper

Surprising findings

• Perfect perceptual quality requires twice the inherent uncertainty for multivariate Gaussian distributions, a counter-intuitive cost of perception.
• Higher-dimensional settings make the uncertainty-perception tradeoff more severe, with small perception gains causing large uncertainty increases.

Limitations

• Uncertainty is estimated via an upper bound (Gaussian-based) because high-dimensional entropy estimation is hard; the bound may be loose.
• Empirical validation limited to super-resolution and inpainting on a few datasets.
• Main results derived for Rényi-1/2 divergence; other divergences only conjectured.

Experimental setup

Datasets: BSD100 SeeTrue

Benchmarks: BSD100 SeeTrue

Metrics: uncertainty (entropy power upper bound) HyperIQA LIQE Q-Align MSE PSNR SSIM

Baselines: EDSR ESRGAN SinGAN SRGAN DIP EnhanceNet DDIM DDPM

Builds on: perception-distortion tradeoff Rényi divergence entropy power inequality Kozachenko-Leonenko estimator

Closest papers by meaning

• Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting (NeurIPS 2026 · similarity 0.90)
• Understanding Hallucinations in Diffusion Models through Mode Interpolation (NeurIPS 2024 · similarity 0.89)
• SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision (AAAI 2026 · similarity 0.89)
• Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models (ICLR 2025 · similarity 0.88)
• On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models (NeurIPS 2025 · similarity 0.88)
• VES-RFT: Rewarding Visual Evidence Sensitivity to Mitigate Hallucinations in Large Vision-Language Models (CVPR 2026 · similarity 0.88)