Research map / Architectures and efficiency / RNNs and normalization
Transformer Memory as a Differentiable Search Index
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace dual encoder with DSI (semantic string docids) | T5-XXL · NQ320K | Hits@1: 24.3 -> 40.4 | Helped |
| replace dual encoder with DSI | T5-Base · NQ10K | Hits@1: 16.2 -> 33.9 | Helped |
| use semantically structured docids instead of naive string docids (ablation) | DSI T5-XXL · NQ320K | Hits@1: 23.8 -> 40.4 | Helped |
| use atomic docids instead of semantic string docids in zero-shot (ablation) | DSI T5-XXL · NQ320K zero-shot | Hits@1 | Helped |
| DSI zero-shot (indexing only) vs BM25 | DSI T5-XXL atomic · NQ320K zero-shot | Hits@1 | Helped |
| DSI vs SentenceT5 in zero-shot | DSI T5-XXL atomic · NQ320K zero-shot | Hits@1 | Helped |
| use atomic docids | DSI · NQ10K/NQ100K/NQ320K | Hits@1: e.g. Large 31.3 / 17.1 / 11.6 | Mixed |
| scale model size from Base to XXL | DSI semantic docids · NQ320K | Hits@1: 27.4 -> 40.4 | Helped |
| scale model size from Base to XXL | Dual Encoder · NQ320K | Hits@1: 20.5 -> 24.3 | Helped |
| remove indexing task (retrieval examples only) (ablation) | DSI naive docid · NQ100K | Hits@1: 13.5 -> 0 | Hurt |
| use Targets2Inputs or span corruption indexing instead of Inputs2Targets (ablation) | DSI naive docid · NQ100K | Hits@1: 13.5 -> 0 | Hurt |
| use Bidirectional instead of Inputs2Targets indexing (ablation) | DSI naive docid · NQ100K | Hits@1: 13.5 -> 13.2 | No clear effect |
Closest papers by meaning
- Semi-Parametric Retrieval via Binary Bag-of-Tokens Index (ICLR 2025 · similarity 0.86)
- LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference (ICLR 2026 · similarity 0.84)
- Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction (NeurIPS 2026 · similarity 0.84)
- Generative Retrieval Meets Multi-Graded Relevance (NeurIPS 2024 · similarity 0.84)
- Bottleneck-Minimal Indexing for Generative Document Retrieval (ICML 2024 · similarity 0.84)
- URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding (AAAI 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.