Research map / Image generation / Autoregressive and latent generation
Towards Scalable Pre-training of Visual Tokenizers for Generation
Jingfeng Yao, Yuda Song, Yucong Zhou, Xinggang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale reconstruction-only AE pre-training compute (ablation) | autoencoder tokenizer · reconstruction vs generation | rFID / gFID: rFID 2.0 -> 0.5; gFID 55.04 -> 58.56 | Hurt |
| add representation-learning losses (VTP) vs AE while scaling tokenizer size 20M to 300M (ablation) | VTP-ViT · ImageNet generation | gFID: 31.28 -> 26.12 | Improved |
| scale pixel decoder (ablation) | VTP · ImageNet generation | gFID: 26.12 -> 24.08 | Improved |
| scale tokenizer data 100K to 100M with VTP (ablation) | VTP-ViT-Large · ImageNet generation | FID: 47.59 -> 27.45 | Improved |
| scale tokenizer data 100K to 100M with AE (ablation) | AE-ViT-Large · ImageNet generation | FID: 58.37 -> 56.71 | No clear effect |
| use f16d64 latent configuration (ablation) | VTP · ImageNet generation | gFID: gFID=27.8; 74.9% linear probing | Improved |
| scale tokenizer from Small to Large (VTP) (ablation) | VTP + LightningDiT · ImageNet 80 epochs | gFID: 5.46 -> 2.81 | Improved |
| scale tokenizer from Small to Large (RAE) (ablation) | RAE + LightningDiT · ImageNet 80 epochs | gFID: 3.50 -> 6.09 | Hurt |
| scale pre-training compute 10x (ablation) | VTP · ImageNet DiT | FID: 65.8% FID improvement | Improved |
| add CLIP loss to tokenizer pre-training (ablation) | VTP · LAION text-to-image | text rendering | Improved |
| add more perception losses (SSL, CLIP) to AE (ablation) | VTP · ImageNet generation | gFID | Improved |
| use smaller reconstruction weight lambda_rec = 0.1 (ablation) | VTP · ImageNet generation | gFID | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- REPA-E — REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers (ICCV 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- UniTok: a Unified Tokenizer for Visual Generation and Understanding (NeurIPS 2025 · similarity 0.92)
- TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.92)
- When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization (NeurIPS 2025 · similarity 0.91)
- Learnings from Scaling Visual Tokenizers for Reconstruction and Generation (ICML 2025 · similarity 0.91)
- End-to-End Vision Tokenizer Tuning (NeurIPS 2025 · similarity 0.91)
- Image Understanding Makes for A Good Tokenizer for Image Generation (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
