Research map / Architectures and efficiency / RNNs and normalization
Learned embeddings for retrieval
Learns sparse, quantizable, and adaptive embeddings for efficient approximate nearest-neighbor search.
76 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in RNNs and normalization, keyword cluster “end · differentiable · translation”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
11 papers in 2023–24 and 26 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 1.7% to 2.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Efficient Indexing of Billion-Scale Datasets of Deep Descriptors (CVPR 2016 · 191 citations)
- 2017 · Definition Modeling: Learning to Define Word Embeddings in Natural Language (AAAI 2017 · 71 citations)
- 2018 · Deep Active Learning for Named Entity Recognition (ICLR 2018 · 381 citations)
- 2019 · Extreme Classification in Log Memory using Count-Min Sketch: A Case Study of Amazon Search with 50M Products (NeurIPS 2019 · 31 citations)
- 2020 · Selection via Proxy: Efficient Data Selection for Deep Learning (ICLR 2020 · 91 citations)
- 2021 · Autoregressive Entity Retrieval (ICLR 2021 · 197 citations)
- 2022 · Transformer Memory as a Differentiable Search Index (NeurIPS 2022 · 80 citations)
- 2023 · SOAR: Improved Indexing for Approximate Nearest Neighbor Search (NeurIPS 2023 · 7 citations)
- 2024 · Bridging the Gap Between End-to-End and Two-Step Text Spotting (CVPR 2024 · 9 citations)
- 2025 · Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations (NeurIPS 2025 · 10 citations)
- 2026 · OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search (ICML 2026 · 1 citation)
Earliest papers here
- Efficient Indexing of Billion-Scale Datasets of Deep Descriptors (CVPR 2016 · 191 citations)
- Efficient Large-Scale Approximate Nearest Neighbor Search on the GPU (CVPR 2016 · 59 citations)
- Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition (NeurIPS 2016 · 50 citations)
What has been tried
54 papers here report 621 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 8%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add sparse IDs | Baidu Industrial Dataset · Recall@800 | 0.3111 -> 0.4466 | Helped | Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations (NeurIPS 2025 · 10 citations) |
| add Bridge | Total-Text · End-to-end accuracy | 77.5 -> 81.5 | Helped | Bridging the Gap Between End-to-End and Two-Step Text Spotting (CVPR 2024 · 9 citations) |
| increase code length from 128 to 256 | DUD-E · AUROC | 83.73 -> 84.96 | Helped | DrugHash: Hashing Based Contrastive Learning for Virtual Screening (AAAI 2025 · 3 citations) |
| vary β from 0.1 to 10 when smoothing the document variance vector | MS MARCO · performance | No clear effect | Distribution-Driven Dense Retrieval: Modeling Many-to-One Query-Document Relationship (AAAI 2025 · 1 citation) | |
| Increase the number of initial search results S | OpenAI Dataset · final cost f | Helped | LotusFilter: Fast Diverse Nearest Neighbor Search via a Learned Cutoff Table (CVPR 2025 · 1 citation) | |
| add diagonal composition and decomposition functions | English sentence-level STS (Score) · Spearman's ρ × 100 | 46.59 ± 0.43 -> 49.68 ± 0.06 | Helped | Banyan: Improved Representation Learning with Explicit Structure (ICML 2025 · 0 citations) |
| Use embedding-space PDF selection instead of random codeword selection | 4-ary IDS code, n=11 · maximum codebook cardinality | +16.8% | Helped | DoDo-Code: an Efficient Levenshtein Distance Embedding-based Code for 4-ary IDS Channel (NeurIPS 2025 · 0 citations) |
| Add in-training retrieved negatives | NQ test (Wiki21m) · top-1 accuracy | 44.9 -> 49.1 | Helped | Semi-Parametric Retrieval via Binary Bag-of-Tokens Index (ICLR 2025 · 0 citations) |
| Use ImplicitSLIM regularization rather than the proposed periodic embedding updates | Collaborative filtering experiments · recommendation performance | No clear effect | ImplicitSLIM and How it Improves Embedding-based Collaborative Filtering (ICLR 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 15 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MS MARCO · 2 of 15 papers
Baselines
- BM25 · 2 of 15 papers
- SPLADE · 2 of 15 papers
- DPR · 2 of 15 papers
- ANCE · 2 of 15 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 37 papers; from OpenAlex and the papers' affiliations).
- Google · 4 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Differentiable optimization layers (125 papers)
- Neural-symbolic reasoning (98 papers)
- Non-autoregressive translation (102 papers)
- Associative memory networks (48 papers)
Elsewhere in the topic
- Efficient network compression and training (229 papers)
- Theory of depth and width (223 papers)
- Representation geometry and collapse (169 papers)
- Dataset distillation and data-efficient training (158 papers)
- Neural scaling laws (155 papers)
- Distributed LLM inference systems (147 papers)
- Layer-wise analysis of language models (145 papers)
- Dataset condensation and flow estimation (126 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).