Research map / Image generation / Autoregressive and latent generation
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RandAR, a GPT-style decoder-only visual autoregressive model trained on random token orders using position instruction tokens, matching raster-order quality on ImageNet while enabling parallel decoding, inpainting, outpainting, and resolution extrapolation.
Problem
Decoder-only autoregressive image generators rely on a predefined raster order, which limits bi-directional context modeling and flexibility for tasks like inpainting and outpainting.
Approach
Insert a learnable position instruction token (shared embedding rotated by 2D RoPE with the target coordinates) before each image token so a causal decoder-only transformer can predict tokens in arbitrary random orders, with minimal architectural change.
Key results
- RandAR-XL (775M) reaches FID 2.25 at 88 steps versus 2.20 for its raster-order counterpart at 256 steps (raster-order 775M counterpart FID 2.16).
- Parallel decoding with 88 steps reduces latency from 16.8s to 6.6s (about 2.5x) for a batch of 64 at 256x256 on A100.
- Random-order training improves SPair-71k PCK per point from 25.8 (first round) to 36.4 with a second pass, while the raster-order model degrades to 3.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with random token orders instead of raster order | RandAR-XL versus the 775M raster-order counterpart · ImageNet 256×256 | FID: 2.16 -> 2.25 | No clear effect |
| Use parallel decoding with 88 steps instead of 256 sequential steps | RandAR · ImageNet 256×256 generation | Latency: 16.8 -> 6.6 sec.; 2.5× acceleration | Helped |
| Use parallel decoding with fewer inference steps | RandAR · ImageNet 256×256 | FID | No clear effect |
| Replace shared RoPE-rotated position embedding with dense learnable embeddings (ablation) | RandAR · ImageNet 256×256 | FID: 2.82 -> 3.07 | Hurt |
| Replace shared RoPE-rotated position embedding with dense learnable embeddings (ablation) | RandAR · ImageNet 256×256 | Inception Score: 293.6 -> 290.6 | Hurt |
| Merge position instruction tokens with image tokens, using dense position embeddings (ablation) | RandAR · ImageNet 256×256 | FID: 2.82 -> 3.37 | Hurt |
| Merge position instruction tokens with image tokens, using dense position embeddings (ablation) | RandAR · ImageNet 256×256 | Inception Score: 293.6 -> 307.6 | Helped |
| Use second-round features by passing the image-token sequence through the model twice | RandAR · ImageNet linear probing | Top-1 accuracy: 57.3 -> 63.1 | Helped |
| Use second-round features by passing the image-token sequence through the model twice | RandAR · SPair71k | PCK (Per Image): 22.1 -> 31.3 | Helped |
| Use hierarchical decoding order for resolution extrapolation | RandAR · ImageNet 512×512 resolution extrapolation | visual consistency | Helped |
| Apply spatial contextual guidance (SCG) | RandAR · ImageNet 512×512 resolution extrapolation | high-frequency detail consistency | Helped |
Surprising findings
- Raster-order decoder-only models degrade sharply under parallel decoding (fewer steps), while RandAR maintains FID up to 88 steps.
- Passing the sequence through a causal random-order model twice yields bi-directional features that improve correspondence, whereas raster-order models fail to benefit.
Limitations
- Zero-shot resolution extrapolation struggles with high-frequency structures such as intricate boundaries.
- Random-order training is a harder task, and RandAR is only slightly behind or comparable to raster counterparts in FID at the largest scale; evaluation limited to ImageNet class-conditional generation.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet 256x256 SPair-71k ImageNet linear probing
Metrics: FID Inception Score precision recall PCK top-1 accuracy latency
Baselines: LlamaGen-L LlamaGen-XL LlamaGen-XXL MaskGIT-re MAR-H VAR RAR-XL DiT-XL SiT-XL Raster-order counterpart
Builds on: LlamaGen MaskGIT MAR 2D RoPE VQGAN
Closest papers by meaning
- ZipAR: Parallel Autoregressive Image Generation through Spatial Locality (ICML 2025 · similarity 0.92)
- HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation (CVPR 2025 · similarity 0.89)
- LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding (ICLR 2025 · similarity 0.89)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.89)
- Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation (NeurIPS 2025 · similarity 0.88)
- Parallelized Autoregressive Visual Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.