Research map / Architectures and efficiency / Quantization
Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the implicit neural residual quantizer QINCo with candidate pre-selection, beam-search encoding, a pairwise additive decoder for search shortlists, and an improved architecture and training recipe.
Problem
Multi-codebook vector quantization (e.g., residual quantization) ignores dependencies between code parts, giving suboptimal rate-distortion; QINCo's encoding is expensive and its decoder too slow for billion-scale search shortlists.
Approach
Speed up QINCo encoding via codeword pre-selection with a learned codebook so beam search becomes affordable, add a pairwise look-up additive decoder built from pairs of QINCo codes to create accurate shortlists, and optimize architecture (decoupled embedding dim, input-output skip) and training (AdamW, cosine schedule, larger batch, dead-codeword reset).
Key results
- On BigANN1M with 16-byte codes, QINCo2 reduces MSE from 0.32e-4 (QINCo) to 0.19e-4, a 34% reduction (0.19 vs 0.32 in the table's units).
- On Deep1M with 8-byte codes, R@1 improves from 36.3% (QINCo) to 45.1%, a 24% relative gain.
- Training with the improved recipe gives over 7x faster training than the QINCo reproduction at similar MSE; pre-selection plus beam search gives roughly 2x speedup over the reproduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QINCo2 instead of QINCo | vector quantization · BigANN1M, 16 bytes | MSE (×10^4): 0.32 -> 0.19 | Helped |
| Use QINCo2 instead of QINCo | vector quantization · Deep1M, 8 bytes | R@1: 36.3% to 45.1% | Helped |
| Add improved training recipe (ablation) | QINCo reproduction · BigANN1M, 8 bytes | training time: 127:38 -> 14:53 | Helped |
| Add improved training recipe (ablation) | QINCo reproduction · BigANN1M, 8 bytes | MSE (×10^4): 1.13 -> 1.14 | No clear effect |
| Add improved architecture (ablation) | QINCo reproduction · BigANN1M, 8 bytes | MSE (×10^4): 1.13 -> 1.09 | Helped |
| Add candidate pre-selection (ablation) | QINCo reproduction · BigANN1M, 8 bytes | training time: 21:42 -> 8:23 | Helped |
| Add candidate pre-selection (ablation) | QINCo reproduction · vector compression | MSE | Mixed |
| Add beam search (ablation) | QINCo reproduction · BigANN1M, 8 bytes | MSE (×10^4): 1.10 -> 0.85 | Helped |
| Increase evaluation beam size (ablation) | QINCo2 · BigANN1M, 8 bytes | MSE (×10^4): 0.85 -> 0.82 | Helped |
| Use 2M optimized code-pairs instead of a single-code additive decoder | approximate decoder for QINCo2 codes · BigANN1M, 8 bytes | R@1: 12.3 -> 28.2 | Helped |
| Use 2M optimized code-pairs rather than 4 consecutive code-pairs | approximate decoder for QINCo2 codes · BigANN1M, 8 bytes shortlist of 10 | R@1: 39.8 -> 46.2 | Helped |
| Use QINCo2-S instead of QINCo | BigANN billion-scale search · BigANN1B | R@1 | Helped |
Surprising findings
- Models with no pre-selection depth (L_s=0, just a learned codebook pre-selection) are Pareto-optimal for encoding times under 1 ms, making a deeper pre-selection network unnecessary.
- Pairwise additive decoder shortlists of size 10 come within 1.7 points R@1 (8-byte) of the full QINCo2 decoder on Deep1M, despite using only about ten network calls versus a million.
Limitations
- Encoding remains expensive relative to plain RQ (beam search cost scales with A*B evaluations)
- AQ and pairwise shortlist decoders still lag the full QINCo2 decoder in R@1 without reranking
- Search experiments exclude the IVF codeword from beam search for simplicity
Experimental setup
Datasets: Deep1B BigANN FB-ssnpp Contriever Deep1M BigANN1M BigANN1B
Benchmarks: BigANN1M Deep1M Contriever1M FB-ssnpp1M BigANN1B Deep1B
Metrics: MSE R@1 R@10 R@100 queries per second latency training time
Baselines: OPQ RQ LSQ UNQ QINCo PQ
Builds on: QINCo residual quantization additive quantization IVF
Closest papers by meaning
- Residual Quantization with Implicit Neural Codebooks (ICML 2024 · similarity 0.96)
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.87)
- NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache (NeurIPS 2025 · similarity 0.86)
- Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression (NeurIPS 2025 · similarity 0.86)
- Learning to Quantize for Training Vector-Quantized Networks (ICML 2025 · similarity 0.86)
- Restructuring Vector Quantization with the Rotation Trick (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.