Research map / Learning theory and algorithms / Prediction and decision losses
Knowledge Distillation for High Dimensional Search Index
Zepu Lu, Jin Chen, Defu Lian, ZAIXI ZHANG, Yong Ge, Enhong Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove distillation loss (ablation) | KDindex (AQ) · MS MARCO Doc | Recall@10: 17.60 -> 18.93 | Improved |
| remove posting list balance (Sinkhorn-Knopp) (ablation) | KDindex (AQ) · MS MARCO Doc | Recall@10: 18.26 -> 18.93 | Improved |
| add balance strategy (ablation) | KDindex (PQ) · MS MARCO Doc | Recall@10: 8.62 -> 8.74 | Improved |
| add distillation loss (ablation) | KDindex (PQ) · MS MARCO Doc | Recall@10: 6.69 -> 8.74 | Improved |
| add distillation loss (ablation) | KDindex (AQ) · MS MARCO Doc | MRR@10: 37.02 -> 41.69 | Improved |
| increase teacher top-K from 10 to 100 | KDindex (AQ) · MS MARCO Doc | Recall@10: 18.93 -> 16.43 | Hurt |
| decrease teacher top-K from 10 to 5 | KDindex (AQ) · MS MARCO Doc | Recall@10: 18.93 -> 18.32 | Hurt |
| differentiable training instead of iterative training | KDindex · SIFT1M, GIST1M, MS MARCO | Recall@10: relative improvement of 1.63% | Improved |
| Distributed-based loss instead of Lambdarank loss | KDindex (AQ) · MS MARCO Doc | Recall@10: 17.60 -> 18.93 | Improved |
| Distributed-based loss instead of Weighted KL loss | KDindex (AQ) · MS MARCO Passage | Recall@10: 11.06 -> 11.19 | No clear effect |
| KDindex distillation on top of AQ | AQ · MS MARCO Doc | Recall@10: 16.73 -> 18.93 | Improved |
| KDindex distillation on top of PQ | PQ · GIST1M | Recall@10: 5.04 -> 9.43 | Improved |
Closest papers by meaning
- Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders (ICLR 2024 · similarity 0.86)
- Learning-Augmented Search Data Structures (ICLR 2025 · similarity 0.85)
- DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models (ICML 2025 · similarity 0.84)
- Semi-Parametric Retrieval via Binary Bag-of-Tokens Index (ICLR 2025 · similarity 0.84)
- MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encoding (NeurIPS 2024 · similarity 0.83)
- Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
