Research map / Image generation / Autoregressive and latent generation

PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: High-resolution diffusion synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PiD, a latent-conditioned pixel diffusion decoder that unifies VAE/RAE latent decoding and 4x/8x super-resolution in one module, distilled to 4 steps with DMD2, enabling early termination of latent diffusion.

Problem

Latent-to-pixel decoders in latent diffusion and autoregressive generators are reconstruction-oriented, cannot synthesize missing high-frequency detail, and become costly at megapixel scale, usually requiring a separate decode-then-upsample cascade.

Approach

Reformulate latent decoding as conditional pixel diffusion: a PixelDiT-based pixel prior is conditioned on noise-corrupted latents via a ControlNet-style adapter with a sigma-aware gate, so decoding and upsampling happen in one generative step, distilled to 4 steps.

Key results

• On FLUX.1 VAE latents, PiD reduces NIQE from 4.04 (best baseline) to 3.50 and runs at about 211 ms with torch.compile on GB200, versus over 1000 ms for diffusion-based SR baselines.
• On SigLIP (Scale-RAE) latents at 8x upsampling, PiD raises Unipercept-IAA from 59.95 to 64.94 and MUSIQ from 73.68 to 74.03.
• PiD decodes 512x512 latents to 2048x2048 in under 1 s with 13 GB peak memory on an RTX 5090 and in 210 ms on GB200; the 4-step student surpasses the 50-step teacher on most IQA metrics.

Surprising findings

• The 4-step distilled student outperforms the multi-step teacher (50 steps) on most no-reference IQA metrics, though the teacher has higher PSNR/SSIM on small-text reconstruction.
• Removing the text-to-image pixel prior severely degrades decoding (NIQE 5.43 to 7.79), showing the high-resolution generative prior is crucial.

Limitations

• Pixel diffusion prior training requires large compute (128 H100 GPUs for about a day), and 4K training uses 96 GB200 GPUs.
• Partially denoised latents can lead to generated content not faithful to the latent (hallucinated details), and PSNR/SSIM fidelity is lower than teacher on text reconstruction.

Experimental setup

Datasets: MultiAspect-4K-1M rendered PDF data internal high-resolution images DPG-Bench prompts ImageNet-1k classes

Benchmarks: DPG-Bench ImageNet-1k

Metrics: MUSIQ (paq2piq) NIQE DEQA MANIQA Q-Align Unipercept IAA Unipercept IQA VisualQuality-R1 latency (ms) PSNR

Baselines: VAE Dec. + Real-ESRGAN VAE Dec. + SeedVR2-3B VAE Dec. + TSD-SR VAE Dec. + InvSR-1 SSDD + Real-ESRGAN SSDD + SeedVR2-3B SSDD + TSD-SR LUA

Builds on: PixelDiT DMD2 ControlNet NTK-aware RoPE SSDD rectified flow

Closest papers by meaning

• Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.90)
• UltraDiff:Transferring High-Fidelity Priors to Compressed Latent Spaces for High-resolution Image Generation (NeurIPS 2026 · similarity 0.89)
• PixelDiT2: Representation-Grounded Pixel Diffusion Transformers (NeurIPS 2026 · similarity 0.89)
• OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates (NeurIPS 2025 · similarity 0.88)
• Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images (AAAI 2024 · similarity 0.88)
• PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis (ICLR 2024 · similarity 0.88)