Research map / Architectures and efficiency / RNNs and normalization
Efficient Training of Retrieval Models using Negative Cache
Erik Lindgren, Sashank J. Reddi, Ruiqi Guo, Sanjiv Kumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use negative cache with 8k negatives instead of 256 sampled negatives | Dual encoder · MS MARCO dev | MRR@10: 0.273 -> 0.310 | Improved |
| Increase cache size from 8k to 2M negatives | Negative Cache · MS MARCO dev | MRR@10: 0.310 -> 0.331 | Improved |
| Increase cache size from 8k to 512k negatives | Negative Cache · TREC 2019 | NDCG@10: 0.600 -> 0.649 | Improved |
| Increase cache size from 512k to 2M negatives | Negative Cache · TREC 2019 | NDCG@10: 0.649 -> 0.630 | Hurt |
| Increase sampled negatives from 16 to 256 | Sample Cross-Entropy · MS MARCO dev | MRR@10: 0.235 -> 0.273 | Improved |
| Negative cache (2M) instead of DPR BM25 hard negatives | Dual encoder · MS MARCO dev | MRR@10: 0.311 -> 0.331 | Improved |
| Negative cache (2M) instead of ANCE 8M-negative training | Dual encoder · MS MARCO dev | MRR@10: 0.330 -> 0.331 | No clear effect |
| Negative cache with 2M negatives instead of DPR | Dual encoder · Natural Questions | Recall@20: 0.784 -> 0.784 | No clear effect |
| Negative cache with 2M negatives instead of DPR | Dual encoder · Natural Questions | Recall@100: 0.854 -> 0.856 | Improved |
| ANCE (20M negatives) versus negative cache (2M) | Dual encoder · Natural Questions | Recall@20: 0.819 -> 0.784 | Hurt |
| Increase cached negatives from 8k to 512k | Negative Cache · Runtime | steps/sec: 3.38 -> 3.32 | No clear effect |
| Use 2M cache with 512 updates instead of 512k cache with 256 updates | Negative Cache · Runtime | steps/sec: 3.32 -> 2.04 | Hurt |
Closest papers by meaning
- A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks (ICML 2024 · similarity 0.87)
- TriSampler: A Better Negative Sampling Principle for Dense Retrieval (AAAI 2024 · similarity 0.87)
- Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders (ICLR 2024 · similarity 0.86)
- USTAD: Unified Single-model Training Achieving Diverse Scores for Information Retrieval (ICML 2024 · similarity 0.85)
- Mitigating the Impact of False Negative in Dense Retrieval with Contrastive Confidence Regularization (AAAI 2024 · similarity 0.85)
- A Gradient Accumulation Method for Dense Retriever under Memory Constraint (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
