Research map / Image generation / Text-to-image diffusion
Unleashing the Potential of Large Language Models for Text-to-Image Generation Through Autoregressive Representation Alignment
Xing Xie, Jiawei Liu, Ziyue Lin, Huijie Fan, Zhi Han, Yandong Tang, Liangqiong Qu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align to interleaved <HYBNEXT> token instead of fixed <REP> token (ablation) | ARRA (Chameleon 7B) · MIMIC-CXR | FID: 6.26 -> 5.30 | Improved |
| align to interleaved <HYBNEXT> token instead of fixed <REP> token (ablation) | ARRA-Adapt (Lumina-mGPT 7B) · MIMIC-CXR | FID: 4.85 -> 4.15 | Improved |
| add global visual alignment loss (ablation) | ARRA (Chameleon 7B) · MIMIC-CXR | FID: 7.11 -> 5.30 | Improved |
| add global visual alignment loss (ablation) | ARRA-Adapt (Lumina-mGPT 7B) · MIMIC-CXR | FID: 5.10 -> 4.15 | Improved |
| use CLS token aggregation instead of average pooling (ablation) | ARRA (Chameleon 7B) · MIMIC-CXR | FID: 6.56 -> 5.30 | Improved |
| use CLS token alone instead of concatenating CLS and average pooling (ablation) | ARRA (Chameleon 7B) · MIMIC-CXR | FID: 5.93 -> 5.30 | Improved |
| use BioMedCLIP instead of Med-SAM as alignment target (ablation) | ARRA (Chameleon 7B) · MIMIC-CXR | FID: 6.54 -> 5.30 | Improved |
| use Med-SAM instead of BioMedCLIP as alignment target (ablation) | ARRA-Adapt (Lumina-mGPT 7B) · MIMIC-CXR | FID: 4.15 -> 4.08 | Mixed |
| use general CLIP-L instead of BioMedCLIP as alignment target (ablation) | ARRA-Adapt (Lumina-mGPT 7B) · MIMIC-CXR | FID: 4.15 -> 4.63 | Hurt |
| add ARRA alignment to LlamaGen trained from scratch | LlamaGen · ImageNet | FID: 4.33 -> 3.61 (343M) | Improved |
| add ARRA alignment to LlamaGen trained from scratch | LlamaGen 343M · ImageNet | IS: 286.6 -> 262.6 | Hurt |
| add ARRA alignment to LlamaGen trained from scratch | LlamaGen · LAION-COCO 2.4M subset | FID: 12.78 -> 11.67 (343M); 11.88 -> 10.45 (3.1B) | Improved |
Closest papers by meaning
- MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis (AAAI 2025 · similarity 0.90)
- Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation (NeurIPS 2025 · similarity 0.90)
- SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation (CVPR 2025 · similarity 0.89)
- LMFusion: Adapting Pretrained Language Models for Multimodal Generation (NeurIPS 2025 · similarity 0.89)
- Elucidating the design space of language models for image generation (ICML 2025 · similarity 0.89)
- Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
