Research map / Image generation / Autoregressive and latent generation
Towards Sequence Modeling Alignment between Tokenizer and Autoregressive Model
Pingyu Wu, Kai Zhu, Yu Liu, Longxiang Tang, Jian Yang, Yansong Peng, Wei Zhai, Yang Cao, Zheng-Jun Zha
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a causal tokenizer decoder to a bidirectional encoder (ablation) | AliTok-B · ImageNet 256 (w/ cfg) | gFID: 2.96 -> 1.88 | Improved |
| add 16 unconstrained prefix tokens (ablation) | AliTok-B · ImageNet 256 (w/ cfg) | gFID / AR accuracy: gFID 1.88 -> 1.86; accuracy 10.7% -> 9.7% | No clear effect |
| add auxiliary loss on prefix tokens (ablation) | AliTok-B · ImageNet 256 (w/ cfg) | gFID: 1.86 -> 1.82 | Improved |
| add second-stage bidirectional decoder retraining (ablation) | AliTok-B · ImageNet 256 (w/ cfg) | gFID: 1.47 -> 1.44 | Improved |
| add second-stage bidirectional decoder retraining (ablation) | AliTok tokenizer · ImageNet-1K val | rFID: 0.91 -> 0.86 | Improved |
| train AR model 800 epochs instead of 400 (ablation) | AliTok-B · ImageNet 256 (w/ cfg) | gFID: 1.61 -> 1.44 | Improved |
| train AR model 800 epochs instead of 400 (ablation) | AliTok-L · ImageNet 256 (w/ cfg) | gFID: 1.49 -> 1.38 | Improved |
| enlarge codebook from 4096 to 16384 (ablation) | AliTok-XL (400 epochs) · ImageNet 256 (w/ cfg) | gFID: 1.28 -> 1.30 | No clear effect |
| enable KV-cache in sampling (ablation) | AliTok-XL · ImageNet 256 | sampling time (s): 107.76 -> 10.07 | Improved |
| make encoder causal instead of bidirectional encoder with causal decoder (ablation) | tokenizer · ImageNet reconstruction | rFID: 0.98 -> 1.56 | Hurt |
Closest papers by meaning
- Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots (ICML 2025 · similarity 0.89)
- Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Generation (NeurIPS 2025 · similarity 0.89)
- FlexTok: Resampling Images into 1D Token Sequences of Flexible Length (ICML 2025 · similarity 0.87)
- ImageFolder: Autoregressive Image Generation with Folded Tokens (ICLR 2025 · similarity 0.87)
- Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective (NeurIPS 2024 · similarity 0.87)
- Parallelized Autoregressive Visual Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
