Research map / Label-efficient and robust learning / Recognition and long-tail
RouterRetriever: Routing over a Mixture of Expert Embedding Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RouterRetriever, a mixture of LoRA-based domain-expert embedding models over a frozen Contriever base, routed per query by average similarity to clustered pilot embeddings, outperforming MSMARCO and multi-task retrievers on BEIR without retraining the router.
Problem
General-purpose single embedding models trained on MSMARCO underperform domain-specific retrievers on specialized domains, and multi-task training is costly to update and degrades unevenly as domains are added.
Approach
Use a frozen shared Contriever base with one LoRA expert per domain, and route each query to the expert whose pilot embeddings (centroids of instances best served by that expert) have the highest average similarity to the query, a training-free routing mechanism designed for retrieval rather than language-model routing.
Key results
- RouterRetriever with seven experts averages 49.6 nDCG@10 over seven in-domain BEIR test sets versus 47.5 for the MSMARCO-only model and 46.4 for multi-task, with the MSMARCO expert included.
- RouterRetriever outperforms the other routing techniques (ExpertClassifierRouter 46.4, ClassificationHeadRouter 46.8, DatasetRouter 48.5) on the same seven datasets.
- On seven unseen BEIR datasets without experts, RouterRetriever averages 31.9 vs 31.6 for MSMARCO-only, while the InstanceOracle reaches 41.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Route queries among domain-specific experts instead of using a single MSMARCO model | Contriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 47.5 -> 49.6 | Helped |
| Route queries among domain-specific experts instead of using a single Multi-Task model | Contriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 46.4 -> 49.6 | Helped |
| Use embedding-similarity routing rather than DatasetRouter | RouterRetriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 48.5 -> 49.6 | Helped |
| Use embedding-similarity routing rather than ExpertClassifierRouter | RouterRetriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 46.4 -> 49.6 | Helped |
| Use embedding-similarity routing rather than ClassificationHeadRouter | RouterRetriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 46.8 -> 49.6 | Helped |
| Add an MSMARCO expert (ablation) | RouterRetriever · BEIR (7 expert datasets) | nDCG@10 (Avg): 49.3 -> 49.6 | Helped |
| Use RouterRetriever instead of a single MSMARCO model on unseen datasets | Contriever · BEIR (7 unseen datasets) | nDCG@10 (Avg): 31.6 -> 31.9 | Helped |
| Add the HotpotQA expert to AR, NF, SF, and FI experts (ablation) | RouterRetriever · ArguAna | nDCG@10: 40.1 -> 38.5 | Hurt |
| Add the HotpotQA expert to AR, NF, SF, and FI experts (ablation) | RouterRetriever · SciFact | nDCG@10: 76.7 -> 72.2 | Hurt |
| Add the HotpotQA expert to AR, NF, SF, and FI experts (ablation) | RouterRetriever · HotpotQA | nDCG@10: 55.3 -> 59.2 | Helped |
| Add the SciDocs expert to the seven-expert RouterRetriever (ablation) | RouterRetriever · SciDocs | nDCG@10: 14.8 to 16.3 | Helped |
| Add the TREC-COVID expert to the seven-expert RouterRetriever (ablation) | RouterRetriever · TREC-COVID | nDCG@10: 44.9 to 56.2 | Helped |
Surprising findings
- Routing techniques from language modeling (ExpertClassifierRouter, ClassificationHeadRouter) often underperform a single MSMARCO-trained retriever on retrieval.
- Adding experts can cause unstable side-effects at low expert counts, e.g. adding a HotpotQA expert drops SciFact from 76.7 to 72.2, though effects become muted with more experts.
Limitations
- Inference requires two forward passes per query, increasing computation cost.
- Routing gap to InstanceOracle remains large (49.6 vs 57.6 average), and diminishing returns appear as experts are added.
- Adding experts at low counts can cause unstable side-effects on existing domains.
Experimental setup
Datasets: MSMARCO ArguAna HotpotQA NFCorpus SciFact FiQA-2018 Quora SciDocs TREC-COVID BEIR generated queries
Benchmarks: BEIR
Metrics: nDCG@10
Baselines: Single model on MSMARCO Multi-Task ExpertClassifierRouter ClassificationHeadRouter DatasetRouter DatasetOracle InstanceOracle
Builds on: Contriever LoRA BEIR DatasetRouter
Closest papers by meaning
- RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models (NeurIPS 2025 · similarity 0.87)
- Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models (ICLR 2025 · similarity 0.87)
- Lookahead Routing for Large Language Models (NeurIPS 2025 · similarity 0.86)
- RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models (NeurIPS 2024 · similarity 0.85)
- Your Mixture-of-Experts LLM Is Secretly an Embedding Model for Free (ICLR 2025 · similarity 0.85)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.