Research map / Image generation / Autoregressive and latent generation

L2P: Unlocking Latent Potential for Pixel Generation

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: High-resolution diffusion synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Transfers a pre-trained latent diffusion model (Z-Image) to pixel space by discarding the VAE, freezing its intermediate DiT layers, training only shallow pixel input/output layers on LDM-generated synthetic images, and enabling native 4K generation.

Problem

Training state-of-the-art pixel-space text-to-image diffusion models from scratch needs prohibitive compute and data, and VAE-based latent diffusion models lose high-frequency detail and hit memory limits at ultra-high resolution.

Approach

Reuse a pre-trained LDM's DiT as a frozen backbone, replace the VAE with large-patch pixel tokenization plus a lightweight Detailer Head (adapted from DiP), train only the shallow layers on synthetic images generated by the source LDM, and raise the noise shift with larger patches to make native 4K training feasible.

Key results

• DPG-Bench average 86.00 for L2P, slightly above source Z-Image-turbo (84.86) and best among pixel models (PixNerd 82.65, DeCo 81.25).
• GenEval overall 0.76 for L2P, below DeCo (0.86) and PixelGen (0.79); the paper reports ~93% of source model performance.
• On UltraHR-eval4k, L2P achieves FID 33.46 and FID-patch 21.77, the lowest among compared 4K methods (best prior FID 35.60 Pixart-sigma).

Surprising findings

• Full-layer tuning stagnates and degrades quality, whereas tuning only shallow layers improves steadily.
• Synthetic LDM-generated data substantially outperforms a real natural-image subset (UltraHR-100K) in convergence and quality.

Limitations

• Performance upper-bounded by the source LDM's priors since training data is synthetic from that model.
• GenEval score (0.76) trails some pixel baselines such as DeCo (0.86) and PixelGen (0.79).
• Does not explore task-specific pixel-level losses for downstream applications.

Experimental setup

Datasets: LDM-generated synthetic images (Z-Image, 10k prompts, 20k images) UltraHR-100K UltraHR-eval4k

Benchmarks: DPG-Bench GenEval UltraHR-eval4k

Metrics: DPG-Bench global/entity/attribute/relation/other/average GenEval overall FID FID-patch Inception Score CLIP score Long CLIP FG-CLIP LPIPS

Baselines: FLUX.1 [Dev] SD3 Medium Qwen-Image Seedream 3.0 Z-Image-turbo PixelFlow PixelGen DeCo PixNerd PixelDiT I-Max HiFlow

Builds on: Z-Image DiP Diffusion Transformer PixelGen PixNerd

Closest papers by meaning

• PixelDiT2: Representation-Grounded Pixel Diffusion Transformers (NeurIPS 2026 · similarity 0.90)
• Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.89)
• RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation (NeurIPS 2025 · similarity 0.88)
• Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers (ICML 2024 · similarity 0.88)
• PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion (NeurIPS 2026 · similarity 0.88)
• Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping (ICLR 2024 · similarity 0.87)