Research map / Image generation / Super-resolution

When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution

NeurIPS 2026 · Image generation › Super-resolution · approach: Diffusion-based super-resolution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PGSR, a pixel-grounded latent flow-matching SR framework that injects pre-VAE LR pixel features into both the ControlNet condition (trajectory guidance) and the frozen VAE decoder (decoder-side grounding) on FLUX.1-dev, training only lightweight modules.

Problem

Generative super-resolution with latent diffusion transformers loses fidelity to the LR input because the VAE compression bottleneck discards fine-grained spatial pixel evidence, producing hallucinated details.

Approach

Extract pixel features from the upsampled LR image before VAE encoding and reuse them at two stages: a gated residual fusion into the latent condition that guides the flow trajectory, and zero-initialized decoder-side feature modulation that injects multi-scale pixel taps into the frozen VAE decoder.

Key results

• On DIV2K-Val x4, PGSR reaches PSNR 25.89 and LPIPS 0.2104, the best among compared methods, with FID 24.88 vs. 27.39 for the next best (SinSR).
• On DRealSR, PGSR obtains the best PSNR (25.97), DISTS (0.0911), FID (46.15), and NIQE (3.8707).
• Removing pixel-grounded guidance on RealSR drops PSNR from 24.75 to 22.15 (ablation table); the sparse attention variant cuts inference time from 61.81 to 38.95 s/img at 1792x1792 with a small PSNR loss.

Surprising findings

• The Stable Diffusion and FLUX VAEs already lose substantial structure on ground-truth HR reconstruction, so decoder-side grounding is motivated by VAE loss even with perfect input.
• PGSR's perceptual metrics on DIV2K-Val and RealSR are not uniformly best (e.g., MANIQA and CLIP-IQA trail DiffBIR and OSEDiff), and it lags in SSIM on DRealSR.

Limitations

• Training requires substantial memory due to large ControlNet and backbone; inference is slower than one-step SR models.
• Mean-error objectives can yield slightly smoother textures than perceptual-sharpness-oriented methods.
• Sparse attention variant slightly degrades quality.

Experimental setup

Datasets: DF2K DIV2K Flickr2K LSDIR FFHQ OST

Benchmarks: DIV2K-Val RealSR DRealSR

Metrics: PSNR SSIM LPIPS DISTS FID NIQE MUSIQ MANIQA CLIP-IQA

Baselines: ResShift StableSR DiffBIR SeeSR PASD OSEDiff SinSR PiSA-SR

Builds on: FLUX.1-dev ControlNet Real-ESRGAN degradation LoRA

Closest papers by meaning

• Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.90)
• Effective Diffusion Transformer Architecture for Image Super-Resolution (AAAI 2025 · similarity 0.89)
• Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.89)
• SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.89)
• RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution (AAAI 2025 · similarity 0.89)
• Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach (CVPR 2025 · similarity 0.89)