Research map / Image generation / Text-to-image diffusion
Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirically studies scaling of autoregressive text-to-image models along discrete vs continuous tokens and raster vs random generation order, and scales a random-order continuous-token model (Fluid) to 10.5B parameters.
Problem
Autoregressive visual models have not scaled as well as LLMs; unclear whether scaling laws hold for text-to-image quality and alignment beyond validation loss, and which token representation and generation order scale best.
Approach
Combine continuous tokens (modeled with a per-token diffusion loss head) with random-order, BERT-style bidirectional masked generation in a text-conditioned transformer; compare against raster-order GPT-style and discrete VQ variants across model sizes, finding continuous random-order scales best on FID and GenEval.
Key results
- Fluid 10.5B achieves zero-shot FID of 6.16 on MS-COCO 30K, the best reported in the paper's comparison table.
- Fluid 10.5B reaches a GenEval overall score of 0.69, versus 0.68 for SD3 8B and 0.67 for DALL-E 3 in the table.
- Fluid 369M matches Parti 20B's FID of 7.23 on MS-COCO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale random-order continuous-token Fluid from 369M to 10.5B parameters | Fluid · MS-COCO zero-shot FID-30K | FID: 7.23 -> 6.16 | Helped |
| Scale random-order continuous-token Fluid from 369M to 3.1B parameters | Fluid · GenEval | overall score: 0.62 -> 0.70 | Helped |
| Scale random-order continuous-token Fluid from 3.1B to 10.5B parameters | Fluid · GenEval | overall score: 0.70 -> 0.69 | No clear effect |
| Use continuous rather than discrete tokens | Autoregressive text-to-image models · Text-to-image generation | visual quality | Helped |
| Use random-order generation with bidirectional attention rather than raster order with causal attention | Autoregressive text-to-image models · GenEval, especially counting and position | GenEval score | Helped |
| Add a 3-layer text aligner instead of using no text aligner (ablation) | 277M Fluid with T5-XL · MS-COCO zero-shot FID | FID: 9.38 -> 8.61 | Helped |
| Add a 6-layer text aligner instead of using no text aligner (ablation) | 277M Fluid with T5-XL · MS-COCO zero-shot FID | FID: 9.38 -> 8.42 | Helped |
| Increase the text aligner from 3 to 6 layers (ablation) | 277M Fluid with T5-XL · MS-COCO zero-shot FID | FID: 8.61 -> 8.42 | Helped |
| Increase training steps and compute | Random-order continuous-token Fluid · Text-to-image generation | FID | Helped |
Surprising findings
- Raster-order discrete-token models plateau in FID and GenEval around 1B parameters, while random-order continuous models keep improving up to 3B.
- Validation loss improvements do not translate to a strict power law for FID or GenEval, though FID and GenEval correlate roughly linearly with validation loss (Pearson 0.917 and -0.931).
Limitations
- Discrete-token models show poor visual quality due to VQ information loss, and scaling does not fix it.
- GenEval scores plateau for 10.5B relative to 3.1B, so the linear correlation with validation loss cannot hold indefinitely.
- Random-order generation prevents kv-caching, making inference less efficient than raster-order models.
Experimental setup
Datasets: WebLI (subset) MS-COCO 2014
Benchmarks: MS-COCO 2014 zero-shot FID-30K GenEval MS-COCO validation loss
Metrics: FID GenEval overall score validation loss visual quality (qualitative) generation speed (sec/img)
Baselines: Parti Muse Imagen DALL-E 2 DALL-E 3 Stable Diffusion 3 Transfusion RAPHAEL Show-o CM3Leon LDM
Builds on: MaskGIT MAR (masked autoregressive with diffusion loss) Muse Parti Imagen
Closest papers by meaning
- Elucidating the design space of language models for image generation (ICML 2025 · similarity 0.89)
- Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Generation (NeurIPS 2025 · similarity 0.89)
- Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots (ICML 2025 · similarity 0.89)
- Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation (NeurIPS 2025 · similarity 0.89)
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.89)
- Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.