Research map / Image generation / Autoregressive and latent generation
Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free, entropy-aware decoding strategy for autoregressive image generation that sets per-token temperature from predicted-logit entropy and extends the idea to speculative-decoding acceptance rules for faster sampling.
Problem
Standard top-K/top-p sampling in autoregressive image generation ignores the low and non-uniform information density of images, causing a trade-off between detail/diversity and structure/text alignment, and speculative decoding wastes compute on low-information tokens.
Approach
Use the entropy of each token's predicted distribution as a proxy for local information density; assign higher temperature to low-entropy tokens and lower temperature to high-entropy tokens, and make speculative decoding's acceptance threshold entropy-dependent. The method applies to next-token, mask-based and scale-wise AR models without training.
Key results
- On LlamaGen, entropy-aware temperature improves FID from 21.94 to 20.36 and DPG from 43.51 to 48.63 on COCO2017 val.
- Combined with SJD speculative decoding, the method reduces average latency on LlamaGen from 26.52s to 22.04s and on Lumina-mGPT from 84.97s to 72.35s, about 15% savings.
- On Meissonic, the entropy-aware temperature lowers FID from 53.61 to 48.37 and raises DPG from 63.83 to 66.19.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use entropy-aware dynamic temperature | LlamaGen · COCO2017 val | DPG: 43.51 -> 48.63 | Helped |
| Use entropy-aware dynamic temperature | LlamaGen · COCO2017 val | FID: 21.94 -> 20.36 | Helped |
| Use entropy-aware dynamic temperature | Lumina-mGPT · COCO2017 val | FID: 29.15 -> 27.44 | Helped |
| Use entropy-aware dynamic temperature | Lumina-mGPT · COCO2017 val | HPSv2.1: 28.92 -> 28.87 | Hurt |
| Add entropy-aware temperature-based masking | Meissonic · COCO2017 val | DPG: 63.83 -> 67.08 | Helped |
| Apply entropy-aware temperature to logits | Meissonic · COCO2017 val | FID: 53.61 -> 48.37 | Helped |
| Add entropy-aware scale-wise temperature | STAR · COCO2017 val | FID: 35.05 -> 32.37 | Helped |
| Add entropy-aware scale-wise temperature | STAR · COCO2017 val | DPG: 70.25 -> 70.86 | Helped |
| Use entropy-aware speculative acceptance instead of SJD | LlamaGen Stage 2 · COCO2017 val | Avg. Latency [s]: 26.52 -> 22.04 | Helped |
| Use entropy-aware speculative acceptance instead of SJD | Lumina-mGPT · COCO2017 val | Avg. Latency [s]: 84.97 -> 72.35 | Helped |
| Use entropy-aware speculative acceptance instead of SJD | LlamaGen Stage 2 · COCO2017 val | FID: 54.49 -> 54.60 | Hurt |
| Use entropy-aware speculative acceptance instead of SJD | Lumina-mGPT · COCO2017 val | FID: 30.76 -> 30.89 | Hurt |
Surprising findings
- Gains are model-dependent: well-trained Lumina-mGPT benefits only marginally, and weak LlamaGen Stage 2 may see little or no gain.
- Entropy is not always a reliable proxy: faces can show low entropy while cluttered backgrounds show high entropy, causing structural distortions in failure cases.
Limitations
- Gains depend on the base model; well-trained or noise-injected models improve little.
- Hyperparameters (T0, alpha, theta) must be tuned per model, and the method is sensitive to them.
- Entropy can misidentify information density (e.g., low entropy on faces), so it may be an imperfect indicator.
Experimental setup
Datasets: MS-COCO 2017 validation MS-COCO 2014
Benchmarks: MS-COCO 2017 val MS-COCO 2014 DPG-bench HPSv2.1
Metrics: FID CLIP-Score DPG HPSv2.1 latency steps
Baselines: Vanilla sampling SJD top-k sampling top-p sampling SDv2.1 PixArt-alpha SDXL SDv3-medium
Builds on: speculative decoding Speculative Jacobi Decoding top-k sampling top-p sampling classifier-free guidance
Closest papers by meaning
- Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation (NeurIPS 2025 · similarity 0.92)
- ZipAR: Parallel Autoregressive Image Generation through Spatial Locality (ICML 2025 · similarity 0.89)
- Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generation (NeurIPS 2025 · similarity 0.89)
- Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching (ICLR 2025 · similarity 0.89)
- LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding (ICLR 2025 · similarity 0.89)
- BitDance: Scaling Autoregressive Generative Models with Binary Tokens (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.