Research map / Image generation / Autoregressive and latent generation
GIVT: Generative Infinite-Vocabulary Transformers
Michael Tschannen, Cian Eastwood, Fabian Mentzer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace discrete VQ tokens with continuous latents modeled by a Gaussian mixture (GIVT-MaskGIT vs MaskGIT) | MaskGIT · ImageNet 512x512 | FID: 7.8007 -> 4.8635 | Improved |
| replace discrete VQ tokens with continuous latents (GIVT-MaskGIT vs MaskGIT) | MaskGIT · ImageNet 256x256 | FID: 4.916 -> 4.64 | Improved |
| add distribution-based classifier-free guidance (DB-CFG) (ablation) | GIVT-Causal · ImageNet 256x256 | FID: 5.6729 -> 3.35 | Improved |
| add adapter (invertible flow) and larger model (ablation) | GIVT-Causal · ImageNet 256x256 | FID: 3.35 -> 2.5933 (GIVT-Causal vs Causal-L+A with DB-CFG) | Improved |
| increase GMM mixtures k from 1 to 16 (ablation) | GIVT-Causal · ImageNet 256x256 | sampling FID | Improved |
| increase VAE KL weight beta (ablation) | beta-VAE with GIVT · ImageNet 256x256 | sampling FID vs reconstruction FID | Mixed |
| model full covariance Gaussian instead of diagonal GMM (ablation) | GIVT-MaskGIT · ImageNet | FID: about 3% gain | No clear effect |
| replace VQ-VAE with beta-VAE and GIVT in UViM | UViM · COCO Panoptic and NYU Depth v2 | PQ and RMSE: PQ 39.0 -> 40.2; RMSE 0.459 -> 0.474 | Mixed |
| GIVT-Causal vs ViT-VQGAN with larger model and 4x longer sequence | unconditional generative model · ImageNet linear probing | accuracy: 65.1 vs 65.1 | No clear effect |
Later papers that compared with this method
3 later accepted papers report GIVT (1.67B) as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- Hyperspherical Latents Improve Continuous-Token Autoregressive Generation (ICLR 2026 · better on 1 of 1 shared result)
- Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation (ICCV 2025 · better on 1 of 1 shared result)
- Autoregressive Image Generation without Vector Quantization (NeurIPS 2024 · better on 1 of 1 shared result)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- MaskGIT — MaskGIT: Masked Generative Image Transformer (CVPR 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- GiT: Towards Generalist Vision Transformer through Universal Language Interface (ECCV 2024 · similarity 0.87)
- Generative Adversarial Transformers (ICML 2021 · similarity 0.86)
- LaVin-DiT: Large Vision Diffusion Transformer (CVPR 2025 · similarity 0.86)
- Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding (ICML 2024 · similarity 0.85)
- Vision Transformers as Probabilistic Expansion from Learngene (ICML 2024 · similarity 0.85)
- SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
