Research map / Large language models / Reasoning and chain of thought
Cost-Aware Contrastive Routing for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSCR, a cost-aware contrastive router that embeds prompts and LLM experts (logit or perplexity fingerprints) in a shared space and routes via FAISS k-NN lookup.
Problem
Cost-aware routing of prompts across diverse, dynamic pools of LLMs, where existing routers ignore prompt context, need expensive profiling, assume fixed experts, or retrain when the pool changes.
Approach
A cost-spectrum InfoNCE loss that treats all correct experts within each cost band as positives, uses band-specific temperatures, and penalizes negatives proportionally to cost; experts are represented by cheap logit footprints (open models) or perplexity fingerprints (APIs) so routing is a nearest-neighbor lookup.
Key results
- On EmbedLLM, CSCR achieves AUDC 0.541 vs 0.515 for UMR and 0.506 for parametric router.
- On RouterBench, CSCR achieves AUDC 0.7110 vs 0.691 for the parametric router and 0.622 for Thompson sampling.
- Cost-spectrum InfoNCE raises AUDC from 0.342 to 0.495 and peak accuracy from 36% to 54% versus vanilla InfoNCE in ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSCR instead of UMR | LLM routing · EmbedLLM | AUDC: 0.515 -> 0.541 | Helped |
| Use CSCR instead of UMR | LLM routing · Mix-Instruct | AUDC: 0.049 -> 0.051 | Helped |
| Use CSCR instead of Parametric routing | LLM routing · RouterBench | AUDC: 0.691 -> 0.7110 | Helped |
| Use perplexity descriptors instead of logit descriptors (ablation) | CSCR · Mix-Instruct | AUDC: 0.461 -> 0.467 | Helped |
| Use perplexity descriptors instead of logit descriptors (ablation) | CSCR · Mix-Instruct | Peak accuracy: less than 0.1% | No clear effect |
| Combine logit and perplexity descriptors in one pool (ablation) | CSCR · Mix-Instruct | AUDC | Helped |
| Replace vanilla InfoNCE with cost-spectrum InfoNCE (ablation) | CSCR · EmbedLLM | AUDC: 0.342 -> 0.495 | Helped |
| Replace vanilla InfoNCE with cost-spectrum InfoNCE (ablation) | CSCR · EmbedLLM | Peak accuracy: 36% to 54% | Helped |
| Add cost bands (ablation) | CSCR · EmbedLLM | AUDC: 0.4574 -> 0.4951 | Helped |
| Add cost bands (ablation) | CSCR · EmbedLLM | Peak accuracy: 0.482 -> 0.540 | Helped |
| Use 5 cost bands rather than fewer or more bands (ablation) | CSCR · Cost-band ablation | AUDC: 0.5480 | Mixed |
| Use 5 cost bands rather than fewer or more bands (ablation) | CSCR · Cost-band ablation | Peak accuracy: 0.590 | Mixed |
Surprising findings
- Vanilla InfoNCE drives the router to over-use cheap experts, which the cost-aware variant corrects at the cost of higher QNC.
- On MixInstruct, the CSCR QNC (9.32) is lower than Oracle's (10.17), while CSCR's peak accuracy matches the best baseline only marginally.
Limitations
- Authors could not test very large LLM pools due to resource limits.
- Perplexity fingerprints approximate black-box expert distributions with GPT-2 scoring, which is an approximation.
- Sample efficiency and descriptor choice (frequency-based tokens) are heuristic.
Experimental setup
Datasets: EmbedLLM MixInstruct RouterBench
Benchmarks: EmbedLLM MixInstruct RouterBench
Metrics: AUDC QNC peak accuracy
Baselines: UMR Thompson Sampling Soft-MoE Parametric softmax router Pareto-optimal routing Random Oracle
Builds on: InfoNCE UMR FrugalGPT FAISS
Closest papers by meaning
- Universal Model Routing for Efficient LLM Inference (ICLR 2026 · similarity 0.90)
- RouteLLM: Learning to Route LLMs from Preference Data (ICLR 2025 · similarity 0.89)
- Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.89)
- RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models (NeurIPS 2024 · similarity 0.88)
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing (ICLR 2024 · similarity 0.88)
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.