Research map / Probabilistic and scientific modeling / Variational and generative sampling
Transporting Quantiles to Codebook: Scalable Vector Quantization without Codebook Collapse
Yuwei Zeng, Zekun Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a scalable vector quantization method that formulates neural quantization as joint modeling and clustering to avoid codebook collapse in VQ-VAEs.
Problem
VQ-VAE discrete representation learning suffers from codebook collapse and inefficient code utilization, which hurts autoregressive generative models that rely on it.
Approach
Reformulates neural quantization as a joint modeling and clustering process (transporting quantiles to the codebook) as a simple alternative to standard vector quantization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace learned/nearest-neighbor VQ codebook with flow-transported quantile codebook | VQ-GAN tokenizer · CIFAR-10 reconstruction (d=14, |C|=16384) | codebook utilization (%): 2.8 -> 100.0 | Improved |
| replace VQ-GAN quantizer with FlowVQ | VQ-GAN tokenizer · CIFAR-10 reconstruction (d=14, |C|=16384) | rFID: 1.65 -> 1.40 | Improved |
| increase codebook size from 1024 to 10240 with fixed latent dim in standard VQ | VQ-GAN · CIFAR-10 reconstruction | codebook utilization (%): 45.6 -> 2.7 | Hurt |
| add auxiliary entropy loss (ablation) | LFQ · CIFAR-10 reconstruction (d=14, |C|=16384) | codebook utilization (%): 0.8 -> 100.0 | Improved |
| add auxiliary entropy loss (ablation) | FlowVQ · CIFAR-10 reconstruction (d=14, |C|=16384) | rFID: 1.40 -> 1.39 | No clear effect |
| add self-correction toward Voronoi centroids (ablation) | FlowVQ transport codebook · CIFAR-10 | FID: 1.49 -> 1.52 | Hurt |
| add self-correction toward Voronoi centroids (ablation) | FlowVQ transport codebook · MP-20 | match rate (%): 88.15 -> 90.81 | Improved |
| add online correction on top of self-correction (ablation) | FlowVQ · CIFAR-10 | FID: 1.52 -> 1.40 | Improved |
| add online correction on top of self-correction (ablation) | FlowVQ · MP-20 | match rate (%): 90.81 -> 90.66 | No clear effect |
| strengthen entropy regularization (epsilon 10 to 1) (ablation) | k-means-initialized OptVQ · MP-20 | match rate (%): 17.85 -> 53.27 | Mixed |
| replace k-means-initialized VQ-VAE with FlowVQ | crystal tokenizer · MP-20 | match rate (%): 78.38 -> 90.81 | Improved |
| replace k-means-initialized VQ-VAE with FlowVQ | robot action tokenizer · DROID action chunks | MSE (e-4): 1.88 -> 1.71 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- FAST+ — FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation (NeurIPS 2023) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Builds on: VQ-VAE
Closest papers by meaning
- Learning to Quantize for Training Vector-Quantized Networks (ICML 2025 · similarity 0.89)
- Finite Scalar Quantization: VQ-VAE Made Simple (ICLR 2024 · similarity 0.89)
- Restructuring Vector Quantization with the Rotation Trick (ICLR 2025 · similarity 0.89)
- Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling (CVPR 2024 · similarity 0.88)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
- Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
