AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Routing and Reasoned Evaluation with Large Language Models

Guiyao Tie, Tianyao Luo, Xueyang Zhou, Chaoran Hu, Yunhong He, Junran Wu, Yuanfan Yao, Pan Zhou, Lichao Sun

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes R$^2$Eval, a routing-aware framework for LLM-as-judge evaluation that allocates evaluation resources across models to address heterogeneous reliability, latency, and token cost.

Problem

LLM-based automated assessment of model outputs faces heterogeneous reliability across judge models, high latency and token costs, and no principled way to allocate evaluation resources.

Approach

A routing-aware evaluation framework (R$^2$Eval) that routes evaluation requests across LLM judges with reasoned evaluation, aiming to balance reliability against cost and latency. Details beyond the abstract are not available.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
route judges with offline ACC-only policy instead of fixed GPT5judge selection · R2Eval routing (average over domains)accuracy (%): 66.0 -> 75.4Improved
route judges with offline ACC-only policy instead of fixed GPT5judge selection · R2Eval routing (average over domains)selected-judge time (s): 41.5 -> 18.7Improved
use ACC-Time policy instead of ACC-onlyoffline routing · R2Eval routing (average over domains)accuracy (%) / time (s): 75.4 -> 71.9 accuracy; 18.7 -> 8.1 timeMixed
add cost penalty (ACC-Time-Cost) to offline routingoffline routing · R2Eval routing (average over domains)accuracy (%) / token cost: 74.1 acc, 2.7 cost vs ACC-only 75.4, 3.0Mixed
use template-only Router 1 for online routingonline routing · R2Eval routing (average over domains)accuracy (%): 66.9 vs ACC-Time 71.9Hurt
use trained Router 3 (Qwen2.5-7B) over ACC-Timeonline routing · R2Eval Coding and Mathaccuracy (%): Coding 63.0 -> 74.6Improved
use Llama-based Router 2online routing · R2Eval Writingaccuracy (%): 77.2Improved
use trained routers instead of ACC-onlyonline routing · R2Eval routing (average over domains)selected-judge latency: 20-45% lowerImproved
use LRM judges instead of LLM judges on reasoning domainsjudges · R2Eval math and logicaccuracy (%): Gemini-2.5-Flash 74.90 reasoning avgImproved
use explicit multi-step reasoning on subjective domainsLRM judges · R2Eval non-reasoningaccuracy (%)No clear effect
longer thinking lengthLRM judges · R2Eval reasoning tasksaccuracy correlation: r = 0.28Mixed
use mid-sized Qwen3-32B instead of strongest LRMsjudges · R2Evallatency and accuracy: latency -35-50%, accuracy within 2-3 ppImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy