Research map / Large language models / Retrieval and knowledge
RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAGRouter, a contrastive-learning router that picks among retrieval-augmented LLMs per query by modeling document-induced knowledge shifts and per-model RAG capability.
Problem
Routing queries to the best LLM when responses are produced under retrieval-augmented generation, where retrieved documents change which model can answer correctly; existing routers ignore documents and assume static parametric knowledge.
Approach
Represents each LLM with a parametric knowledge embedding plus a learnable RAG capability embedding, fuses them with document and query-document cross-encoder features via attention, and trains with cross-setting and intra-setting contrastive learning plus a classification loss so that query-model similarity reflects answerability under RAG.
Key results
- RAGRouter averages 64.46% test accuracy across tasks, versus 60.85% for the best single LLM (Llama-3.3-70B-Instruct) and 61.17% for the best non-RAG-aware router (GraphRouter).
- With closed-source LLMs added on NQ and WebQ, RAGRouter reaches 59.38% average, +1.67% over GPT-4o while invoking GPT-4o on 44.79% of samples.
- Inference takes about 0.011 s per instance on a single RTX 4090D with 4147 MiB peak GPU memory.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RAG-aware routing instead of the best single LLM | RAGRouter · Main tasks (Avg) | testing accuracy (%): 60.85 -> 64.46 | Helped |
| Use RAG-aware routing instead of non-RAG-aware routing | RAGRouter · Main tasks (Avg) | testing accuracy (%): 61.17 -> 64.46 | Helped |
| Include closed-source LLM candidates in the pool | RAGRouter · NQ + WebQ | testing accuracy (%): 57.71 -> 59.38 | Helped |
| Use score-threshold-based routing | RAGRouter · MedMCQA Local | Area (%): 57.12 | Helped |
| Remove the Cross Encoder (equivalently, add query-document interaction encoding) (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.48 -> 64.46 | Helped |
| Remove the RAG Capability Embedding Layer (equivalently, explicitly model RAG capability) (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.70 -> 64.46 | Helped |
| Remove CSC (equivalently, add Cross-Setting Contrast) (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.49 -> 64.46 | Helped |
| Remove ISC (equivalently, add Intra-Setting Contrast) (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.82 -> 64.46 | Helped |
| Remove ISC and CSC (equivalently, use both contrastive objectives) (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 62.28 -> 64.46 | Helped |
| Set embedding dimension to 128 instead of 768 (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.00 -> 64.46 | Helped |
| Set embedding dimension to 256 instead of 768 (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.65 -> 64.46 | Helped |
| Set embedding dimension to 384 instead of 768 (ablation) | RAGRouter · Main tasks (Avg) | testing accuracy (%): 63.43 -> 64.46 | Helped |
Surprising findings
- On average across models and tasks, 30.86% of initially incorrect queries become correct with RAG while 31.46% of initially correct queries become incorrect, indicating frequent RAG-induced reversals.
- Removing intra-setting contrast alone costs 0.64 points while removing cross-setting contrast costs 0.97 points; removing both costs 2.18 points.
Limitations
- Evaluated on small query sets (270 test queries for PopQA/MedMCQA, 240 for NQ/WebQ/TriviaQA), so differences of a few points may be within noise.
- Relies on per-query labels obtained by running all candidate LLMs with and without retrieval, which is costly to collect.
- Latency-aware extension uses prior parameter-size-based ordering and does not explicitly model latency.
Experimental setup
Datasets: PopQA MedMCQA Natural Questions WebQuestions TriviaQA 2018 English Wikipedia dump WikiASP HotpotQA
Benchmarks: PopQA MedMCQA Natural Questions WebQuestions TriviaQA WikiASP HotpotQA
Metrics: Cover Exact Match accuracy Area under accuracy-latency curve Peak accuracy Latency gap-to-match ROUGE-L
Baselines: Prompt LLM GraphRouter RouterDC KNN Router Matrix Factorization Oracle Single Best Random Weighted
Builds on: RouterDC GraphRouter RouteLLM EmbedLLM
Closest papers by meaning
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.89)
- GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation (NeurIPS 2025 · similarity 0.89)
- RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards (ICLR 2025 · similarity 0.89)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.89)
- RAGGED: Towards Informed Design of Scalable and Stable RAG Systems (ICML 2025 · similarity 0.88)
- Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need? (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.