Research map / Architectures and efficiency / RNNs and normalization
Improving the Accuracy of Dense Retrieval on the Quantized Indexes via Gradient Optimization of the Target Embeddings
Cong Tan, Yongqi Shao, Hong Huo, Tao Fang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with gradient-updated cached target embeddings and similarity-based large negatives | ANCE model · MS MARCO Dev (QHNSW index) | MRR@10: 12.74 -> 26.32 | Improved |
| same training | ANCE model · MS MARCO Dev (FlatIP index) | MRR@10: 32.26 -> 29.71 | Hurt |
| same training | ANCE model · TriviaQA Test (QHNSW index) | R@1: 51.22 -> 53.91 | Improved |
| same training | ANCE model · TriviaQA Test (QHNSW index) | R@10: 74.80 -> 74.39 | Hurt |
| same training | DPR model · Natural Questions Test (QHNSW index) | R@1: 46.06 -> 51.52 | Improved |
| same training | DPR model · Natural Questions Test (FlatIP index) | R@1: 52.13 -> 52.49 | No clear effect |
| add similarity-based large negatives retrieved from the quantized index (ablation) | DPR on QHNSW · Natural Questions Test | R@1: 46.06 -> 49.72 | Improved |
| add similarity-based large negatives retrieved from the quantized index (ablation) | DPR on QHNSW · Natural Questions Test | R@100: 84.65 -> 84.12 | Hurt |
| add gradient updates of cached target embeddings on top of large negatives (ablation) | DPR on QHNSW · Natural Questions Test | R@1: 49.72 -> 51.52 | Improved |
| set temperature to 8 versus other values from 0.5 to 16 (ablation) | training · Natural Questions Test (QHNSW) | R@1: 49.2 -> 51.5 | Improved |
Closest papers by meaning
- A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks (ICML 2024 · similarity 0.90)
- NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval (ICLR 2025 · similarity 0.87)
- Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders (ICLR 2024 · similarity 0.87)
- Semi-Parametric Retrieval via Binary Bag-of-Tokens Index (ICLR 2025 · similarity 0.87)
- A Gradient Accumulation Method for Dense Retriever under Memory Constraint (NeurIPS 2024 · similarity 0.86)
- Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
