Research map / Architectures and efficiency / Quantization
Switchable Token-Specific Codebook Quantization For Face Image Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes switchable token-specific codebook quantization (routed codebook groups plus per-token sub-codebooks) for low-bpp face image compression, plugged into TiTok and MaskGIT-VQGAN and evaluated by face recognition accuracy.
Problem
Global shared codebooks in VQ-VAE-style face image compressors degrade sharply at low bits per pixel, hurting reconstruction and face recognition accuracy.
Approach
Replace the single global codebook with multiple codebook groups selected per image by a router (costing only log2 M bits), and further give each token its own sub-codebook, so total codebook capacity grows while per-token index bit-width shrinks.
Key results
- On MaskGIT-VQGAN at 0.0469 bpp, MeanAcc rises from 90.70% to 93.51%; on TiTok-S at 0.0234 bpp, from 87.56% to 91.66%.
- On TiTok-S, bpp drops from 0.0234 to 0.0157 while accuracy stays comparable (87.56% to 87.60%).
- Per-token codebook utilization mean rises from 54.17% (global-shared) to 74.02% on LFW at 0.0234 bpp.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add switchable token-specific codebook quantization to MaskGit-VQGAN | MaskGit-VQGAN · MeanAcc across five face recognition datasets | MeanAcc (%): 90.70 -> 93.51 | Helped |
| Add switchable token-specific codebook quantization to TiTok-S | TiTok-S · MeanAcc across five face recognition datasets | MeanAcc (%): 87.56 -> 91.66 | Helped |
| Add switchable token-specific codebook quantization to TiTok-L | TiTok-L · MeanAcc across five face recognition datasets | MeanAcc (%): 65.07 -> 73.13 | Helped |
| Reduce bitrate with switchable token-specific codebooks | TiTok-S · MeanAcc across five face recognition datasets | MeanAcc (%): 87.56% -> 87.60% | No clear effect |
| Reduce bitrate with switchable token-specific codebooks | MaskGit-VQGAN · MeanAcc across five face recognition datasets | MeanAcc (%): 90.70% -> 92.23% | Helped |
| Replace one 4096-entry codebook with 16 1024-entry codebooks (ablation) | MaskGit-VQGAN · MaskGit-VQGAN, 256 tokens | MeanAcc (%): 93.51 -> 92.23 | Hurt |
| Replace one 4096-entry codebook with 256 256-entry codebooks (ablation) | TiTok-S · TiTok-S, 128 tokens | MeanAcc (%): 91.66 -> 87.60 | Hurt |
| Replace one 4096-entry codebook with 256 256-entry codebooks (ablation) | TiTok-L · TiTok-L, 32 tokens | MeanAcc (%): 73.13 -> 66.02 | Hurt |
| Use codebook routing (CR) instead of nearest-neighbor search (NN) (ablation) | MaskGit-VQGAN with 16 codebooks · MaskGit-VQGAN, 256 tokens | MeanAcc (%): 92.23 -> 92.22 | No clear effect |
| Use codebook routing (CR) instead of nearest-neighbor search (NN) (ablation) | TiTok-S with 256 codebooks · TiTok-S, 128 tokens | MeanAcc (%): 87.60 -> 87.54 | No clear effect |
| Use codebook routing (CR) instead of nearest-neighbor search (NN) (ablation) | TiTok-L with 256 codebooks · TiTok-L, 32 tokens | MeanAcc (%): 66.02 -> 65.65 | Hurt |
| Add token-shared codebooks and codebook routing to the original single-codebook setup (ablation) | Ablation configuration · Ablation configuration | MeanAcc (%): 88.11 -> 88.24 | Helped |
Surprising findings
- Routing-based inference (codebook group chosen by the router, CR) keeps accuracy nearly unchanged while cutting storage substantially versus nearest-neighbor search over all codebooks (e.g., 4194.70 MB to 354.70 MB for MaskGIT-VQGAN).
Limitations
- Performance depends heavily on the underlying autoencoder baseline since no encoder/decoder changes are proposed.
- Increasing the number of codebooks to 128 degrades recognition performance, attributed to learning difficulty.
- Total codebook storage grows substantially with token-specific codebooks.
Experimental setup
Datasets: CASIA-WebFace LFW CFP-FP AgeDB CPLFW CALFW FFHQ
Benchmarks: LFW CFP-FP AgeDB CPLFW CALFW
Metrics: MeanAcc IDS (identity similarity) bpp inference time storage
Baselines: JPEG 2000 CodeFormer MaskGit-VQGAN TiTok-S TiTok-L
Builds on: VQ-VAE VQGAN TiTok Mixture-of-Experts
Closest papers by meaning
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.85)
- Kepler codebook (ICML 2024 · similarity 0.84)
- Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling (CVPR 2024 · similarity 0.84)
- Towards Improved Text-Aligned Codebook Learning: Multi-Hierarchical Codebook-Text Alignment with Long Text (CVPR 2025 · similarity 0.84)
- Towards Unified Human Perception and Machine Understanding: Token Flow Guided Compression Framework (CVPR 2026 · similarity 0.83)
- Image and Video Tokenization with Binary Spherical Quantization (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.