Research map / Image generation / Autoregressive and latent generation
Vector-quantized Image Modeling with Improved VQGAN
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CNN encoder and decoder with ViT encoder and decoder (ablation) | VQGAN · ImageNet reconstruction | FID: 2.26 -> 1.55 | Helped |
| Replace PatchGAN with StyleGAN discriminator (ablation) | ViT-VQGAN Base-Base · ImageNet reconstruction | FID: 3.88 -> 1.55 | Helped |
| Reduce factorized code lookup dimension from 256 to 16 (ablation) | ViT-VQGAN Base-Base · ImageNet reconstruction | FID: 3.68 -> 1.50 | Helped |
| Reduce factorized code lookup dimension from 256 to 8 (ablation) | ViT-VQGAN Base-Base · ImageNet reconstruction | FID: 3.68 -> 1.52 | Helped |
| Remove L2 normalization of encoded and codebook variables (ablation) | ViT-VQGAN Base-Base · ImageNet reconstruction | FID: 1.55 -> 5.44 | Hurt |
| Use the ViT-VQGAN quantizer instead of DALL-E dVAE (ablation) | VIM-Base · ImageNet linear probe | Top-1 accuracy: 63.8 -> 65.1 | Helped |
| Use the ViT-VQGAN quantizer instead of CNN-VQGAN (ablation) | VIM-Base · ImageNet linear probe | Top-1 accuracy: 61.8 -> 65.1 | Helped |
| Use VIM-Large instead of iGPT-L | ImageNet generative-pretraining linear probe · ImageNet | Top-1 accuracy: 60.3 -> 73.2 | Helped |
| Use ViT-VQGAN instead of VQGAN | Class-conditioned image generation · ImageNet 256×256 | FID: 17.04 -> 4.17 | Helped |
| Use ViT-VQGAN instead of VQGAN | Class-conditioned image generation · ImageNet 256×256 | IS: 70.6 -> 175.1 | Helped |
| Use a 32×32 token sequence instead of 16×16 (ablation) | ViT-VQGAN + VIM-Large · ImageNet class-conditioned generation | FID: 5.3 -> 4.17 | Helped |
| Apply classifier-based rejection sampling at acceptance rate 0.5 (ablation) | ViT-VQGAN · ImageNet class-conditioned generation | FID: 4.17 -> 3.04 | Helped |
Closest papers by meaning
- Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis (CVPR 2024 · similarity 0.89)
- Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling (CVPR 2024 · similarity 0.89)
- Scaling the Codebook Size of VQ-GAN to 100,000 with a Utilization Rate of 99% (NeurIPS 2024 · similarity 0.89)
- Channel-wise Vector Quantization (NeurIPS 2026 · similarity 0.89)
- VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.88)
- LG-VQ: Language-Guided Codebook Learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.