Research map / Image generation / Autoregressive and latent generation
Kepler codebook
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Casts VQ codebook training as sphere packing and regularizes codebook token distribution toward an Irwin-Hall prior with codebook partitioning, improving codebook usage, reconstruction, and generation.
Problem
Learned VQ codebooks suffer from codebook collapse (low usage) and unbalanced token training, which hurts image reconstruction, generation, and generalization.
Approach
Derives an ideal codebook distribution from Kepler's conjecture (hexagonal/dense sphere packing), shows each dimension approximates an Irwin-Hall distribution, and uses KL divergence to that prior as regularization together with a partition of encoder outputs to reduce dimensionality.
Key results
- On ADE20K reconstruction, the method reaches 21.34 PSNR and 16.87 rFID versus VQGAN's 18.89 PSNR and 28.17 rFID.
- Codebook usage rises from 35% (VQGAN baseline) to 100% with partitioning plus IHD regularization on ADE20K.
- On DRealSR super-resolution with an LDM, FID improves from 35.66 to 26.11 when using the Kepler autoencoder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Kepler codebook instead of VQ-VAE | image reconstruction and generation · ADE20K | PSNR[R]: 19.95 -> 21.34 | Helped |
| use Kepler codebook instead of VQGAN | image reconstruction and generation · ADE20K | rFID[R]: 28.17 -> 16.87 | Helped |
| use Kepler codebook instead of Reg-VQ | image reconstruction and generation · ADE20K | FID[G]: 34.47 -> 33.84 | Helped |
| use Kepler codebook instead of Reg-VQ | image reconstruction and generation · CelebA-HQ | PSNR[R]: 22.05 -> 24.85 | Helped |
| use Kepler codebook instead of Reg-VQ | image reconstruction and generation · CelebA-HQ | rFID[R]: 10.09 -> 8.59 | Helped |
| use Kepler codebook instead of Reg-VQ | image reconstruction and generation · CelebA-HQ | FID[G]: 15.34 -> 14.96 | Helped |
| replace LDM autoencoder with Kepler codebook model | LDM image reconstruction · ImageNet | PSNR: 27.48 -> 28.00 | Helped |
| replace LDM autoencoder with Kepler codebook model | LDM super-resolution · DRealSR | PSNR: 24.86 -> 26.15 | Helped |
| add IHD regularization to VQGAN (ablation) | VQGAN · ADE20K | rFID: 28.17 -> 26.28 | Helped |
| add IHD regularization to VQGAN (ablation) | VQGAN · ADE20K | codebook usage: 35% -> 40% | Helped |
| add codebook partition to VQGAN (ablation) | VQGAN · ADE20K | rFID: 28.17 -> 19.34 | Helped |
| add codebook partition to VQGAN (ablation) | VQGAN · ADE20K | codebook usage: 35% -> 100% | Helped |
Surprising findings
- IHD regularization alone gives only a slight gain; the benefit is largely realized when combined with codebook partitioning, since KL estimation is hard in high dimensions.
Limitations
- The Kepler-derived hexagonal distribution rests on preconditions and a relaxed objective (equal basis-vector angles), so optimality is not fully established.
- Evaluation is mostly on ADE20K and CelebA-HQ with a VQGAN-style architecture at 256x256 resolution.
Experimental setup
Datasets: ADE20K CelebA-HQ ImageNet DRealSR
Benchmarks: ADE20K CelebA-HQ MS-COCO LSDIR DIV2K ImageNet DRealSR
Metrics: PSNR rFID FID SSIM LPIPS codebook usage
Baselines: VQ-VAE VQGAN Reg-VQ FSQ LDM
Builds on: VQGAN Reg-VQ VQ-VAE latent diffusion model
Closest papers by meaning
- Efficient Image Synthesis with Sphere Latent Encoder (NeurIPS 2026 · similarity 0.84)
- Switchable Token-Specific Codebook Quantization For Face Image Compression (NeurIPS 2025 · similarity 0.84)
- dRAE: Representation Autoencoder with Hyper-Spherical Codes (NeurIPS 2026 · similarity 0.83)
- Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling (CVPR 2024 · similarity 0.82)
- BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities (ICLR 2025 · similarity 0.82)
- SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.