Research map / Architectures and efficiency / Quantization

HyperVQ: Enabling Hyperprior Entropy Modeling for VQ-Based Generative Image Compression

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HyperVQ, a hyperprior entropy model that predicts a Gaussian in the codebook embedding space and converts distances to anchors into categorical index probabilities, plugged into several VQ codecs to cut bitrate.

Problem

VQ-based generative image compression uses static frequency tables for index entropy coding and lacks a differentiable, content-adaptive index rate term, so RD is not jointly optimized end-to-end.

Approach

Predict an isotropic Gaussian (mean and sigma) in the D-dimensional codebook embedding space via a hyperprior; treat codebook entries as anchors and map their Gaussian densities to a softmax categorical distribution over indices, yielding a differentiable cross-entropy rate term.

Key results

• HyperVQ averages 18.5% bitrate saving over theoretical bpp across VQ codecs, versus 2.54% for Huffman coding (7.28x).
• HVQ-CGIC reaches 0.493 BPP on Kodak (-21.1% vs. no compression) in the Stage B setting, versus 0.594 for Control-GIC.
• HVQ-CGIC (RD) achieves LPIPS 0.0197 on Kodak versus 0.0248 for Control-GIC at the same downsampling factor, and BD-Rate LPIPS savings of -87.30% vs. M&S.

Surprising findings

• The Huffman coding gain is near zero for several codecs (e.g., -0.2% for VQGAN-LC at 100K codebook), while HyperVQ gains are substantial.
• The HVQ-CGIC (BPP) variant with frozen backbone reaches a lower bpp than the RD-tuned variant but worse LPIPS (0.0278 vs 0.0197).

Limitations

• Isotropic covariance assumption limits cross-dimensional correlation modeling.
• Centroid-density approximation to Voronoi cell mass is only bounded under small-cell and non-small-sigma conditions.
• Ultra-large codebooks rely on a bounded tail-floor approximation for latency, with a small accuracy gap to exact softmax.

Experimental setup

Datasets: ImageNet-1K

Benchmarks: ImageNet-1K Val Kodak CLIC 2020 DIV2K

Metrics: bpp bitrate gain LPIPS DISTS FID NIQE IS PSNR BD-Rate encode/decode time

Baselines: Huffman coding BPG VVC M&S HiFiC CDC SCR CTC Control-GIC MS-ILLM CRDR EGIC

Builds on: hyperprior VQGAN Control-GIC McQuic VQGAN-LC

Closest papers by meaning

• Generative Latent Coding for Ultra-Low Bitrate Image Compression (CVPR 2024 · similarity 0.88)
• Finite Scalar Quantization: VQ-VAE Made Simple (ICLR 2024 · similarity 0.87)
• Efficient Generative Modeling with Residual Vector Quantization-Based Tokens (ICML 2025 · similarity 0.87)
• Diversify, Contextualize, and Adapt: Efficient Entropy Modeling for Neural Image Codec (NeurIPS 2024 · similarity 0.87)
• Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation (ICLR 2025 · similarity 0.87)
• Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.86)