Research map / Architectures and efficiency / Quantization
Residual Quantization with Implicit Neural Codebooks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QINCo, a neural residual quantizer that generates step-specific codebooks conditioned on prior reconstructions, and combines it with IVF and approximate AQ-decoder re-ranking for large-scale search.
Problem
Standard residual quantization uses a fixed codebook per step, which is suboptimal because residual distributions depend on previously selected codewords; the goal is more accurate multi-codebook vector compression and search.
Approach
Instead of fixed per-step codebooks, a small residual MLP network per step generates specialized codebooks conditioned on the partial reconstruction and a base codebook (initialized from pretrained RQ), transforming codebook vectors rather than the data vectors, so no gradient estimator is needed.
Key results
- On BigANN1M 16 bytes, QINCo reaches MSE 0.32e4 and R@1 71.9% versus RQ 1.30e4 / 49.0% and UNQ 0.57e4 / 59.3%.
- On Deep1M 8 bytes, QINCo achieves R@1 36.3% versus LSQ 24.6% and UNQ 26.7%.
- On SIFT1M with 4 bytes, QINCo reaches R@1 14.9% versus DeepQ 11.0% and RVPQ 10.2%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt codebooks to residual distributions with implicit neural codebooks | Residual quantization · BigANN1M, 8 bytes | MSE (×10^4): 2.49 -> 1.12 | Helped |
| Adapt codebooks to residual distributions with implicit neural codebooks | Residual quantization · BigANN1M, 8 bytes | R@1: 27.9 -> 45.2 | Helped |
| Use 10M rather than 500k training vectors | QINCo · BigANN1M, 8 bytes | R@1: 40.2 -> 45.2 | Helped |
| Use only the loss after the final quantization step instead of losses at all steps (ablation) | QINCo · Quantization performance | MSE | Hurt |
| Detach the losses so each step updates only its own parameters (ablation) | QINCo · Quantization performance | MSE | Hurt |
| Detach the losses so each step updates only its own parameters (ablation) | QINCo · 8 bytes Deep1B encoding | R@1 | Mixed |
| Share the concatenation-block parameters across quantization steps (ablation) | QINCo · Quantization performance | MSE | Hurt |
| Share the residual-MLP parameters across quantization steps (ablation) | QINCo · Quantization performance | MSE | Hurt |
| Share both concatenation-block and residual-MLP parameters across quantization steps (ablation) | QINCo · Quantization performance | MSE | Hurt |
| Increase model capacity with too few training vectors (ablation) | QINCo · 500k-vector training | Accuracy | Hurt |
| Replace QINCo's first affine layer with a low-rank projection (ablation) | QINCo-LR · Contriever embeddings, 8 bytes | MSE: 1.45 -> 1.46 | Hurt |
| Increase training data from 500k to 10M vectors (ablation) | OPQ · BigANN1M, 8 bytes | MSE (×10^4): 2.95 -> 2.99 | No clear effect |
Surprising findings
- Truncated (prefix) QINCo codes perform on par with models trained for that code length, enabling dynamic rates from one model.
- Training QINCo with only the last-step loss drastically reduces performance and makes optimization unstable.
Limitations
- Encoding is considerably slower than conventional RQ/PQ and competing methods.
- Parameter count scales as O(D^2), motivating a low-rank variant for high-dimensional data.
Experimental setup
Datasets: Deep1B BigANN1B BigANN1M Deep1M Contriever1M FB-ssnpp1M SIFT1M
Benchmarks: BigANN1M Deep1M Contriever1M FB-ssnpp1M SIFT1M BigANN1B Deep1B
Metrics: MSE R@1 R@10 R@100 queries per second codeword entropy
Baselines: OPQ RQ LSQ UNQ RVPQ DeepQ IVF-PQ IVF-RQ
Builds on: residual quantization UNQ Faiss HNSW Adam
Closest papers by meaning
- Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks (ICLR 2025 · similarity 0.96)
- Restructuring Vector Quantization with the Rotation Trick (ICLR 2025 · similarity 0.87)
- Learning to Quantize for Training Vector-Quantized Networks (ICML 2025 · similarity 0.86)
- NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache (NeurIPS 2025 · similarity 0.86)
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.86)
- Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.