Research map / Image generation / Text-to-image diffusion

Denoising Autoregressive Transformers for Scalable Text-to-Image Generation

ICLR 2025 · Image generation › Text-to-image diffusion · approach: Compositional text-to-image guidance

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DART, a non-Markovian diffusion model implemented as an autoregressive transformer that denoises continuous VAE patch tokens across noise levels, with token-level AR (DART-AR) and flow-matching (DART-FM) variants, evaluated on ImageNet class-conditional and CC12M text-to-image generation.

Problem

Standard Markovian diffusion models cannot exploit the full generation trajectory, causing training and inference inefficiency; autoregressive image models either need quantization or fail to leverage progressive denoising.

Approach

Replace the Markovian denoiser with a non-Markovian generator: each denoising step conditions on all previous noisy versions via a chunk-causal Transformer trained in parallel over noise levels, with optional token-level AR decoding and a flow-matching refinement head, all on continuous (unquantized) latents.

Key results

• DART-AR achieves FID 3.98 on ImageNet 256x256 (IS 256.8) and DART-FM 3.82 (IS 263.8), versus vanilla DART at 5.62 with 16 steps.
• On text-to-image, DART-FM reaches FID 11.12, improving over vanilla DART.
• Reproduced DiT with 16 steps gets FID 19.52 while DART at 16 steps gets 5.62 on ImageNet, though DART still trails VAR (1.73) and MAR (1.55).

Measured effects reported in the paper

Surprising findings

• DART-AR with 4096 sequential AR steps has much higher wall-clock inference time, since its AR steps are not well parallelized in the current implementation.

Limitations

• Non-Markovian modeling memory grows with number of noise levels T, limiting token count and text-to-image capacity.
• Still a gap to SOTA visual generators such as VAR and MAR on ImageNet, and baselines were trained with more compute.
• DART-AR inference is expensive due to K x T sequential steps.

Experimental setup

Datasets: ImageNet CC12M MSCOCO 2017

Benchmarks: ImageNet 256x256 CC12M MSCOCO 2017 validation

Metrics: FID IS CLIP score

Baselines: DiT SiT MAR MAR-AR VAR LDM ADM VQGAN RQ-Transformer CDM

Builds on: DiT VDM MAR Kaleido DDPM

Closest papers by meaning

• Beyond Text-to-Image: Liberating Generation with a Unified Discrete Diffusion Model (ICLR 2026 · similarity 0.92)
• Denoising Autoregressive Representation Learning (ICML 2024 · similarity 0.89)
• FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute (CVPR 2025 · similarity 0.89)
• Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis (CVPR 2024 · similarity 0.89)
• Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.89)
• Fast constrained sampling in pre-trained diffusion models (NeurIPS 2025 · similarity 0.89)