Research map / Large language models / Reasoning and chain of thought

Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies reasoning vs non-reasoning LLM judges and proposes RACER, a KL-DRO constrained routing policy that chooses between reasoning and instruct judge modes under a cost budget.

Problem

Reasoning-capable LLMs as judges improve accuracy on verification-heavy tasks but at high token cost and not uniformly; routers trained on static data may violate budgets or misroute under distribution shift.

Approach

Formulates reasoning-vs-instruct routing as a constrained distributionally robust optimization with a KL uncertainty set over preference data, solved via closed-form reweighting and primal-dual updates with entropy regularization, with linear last-iterate convergence guarantees.

Key results

• RACER reaches 72.2/85.8/90.0% accuracy at 1.7B/4B/8B, beating the strongest routing baseline (M-IRT) by 0.64, 1.10 and 1.06 points at budget C=4.
• At roughly half of All-Reasoning cost, RACER matches or surpasses All-Reasoning accuracy on training and in-distribution splits.
• Proves a unique saddle point and linear last-iterate convergence of the router policy in KL divergence.

Surprising findings

• Reasoning gives limited or negative gains on safety and knowledge judgments, despite large gains on math and coding.
• Reasoning and non-reasoning judges often agree, so non-reasoning is preferable when they agree; headroom shrinks with model size.

Limitations

• Binary routing only, with a KL uncertainty set; large uncertainty causes conservative routing that underuses reasoning.
• Convergence guarantees are stated for the convex/policy-class setting; parameterized (neural) extension holds only up to approximation error.
• Cost is measured as token count ratio, which may not reflect latency or monetary cost.

Experimental setup

Datasets: Skywork Reward Preference Dataset Math-Step-DPO-10K Code-Preference-Pairs Magpie Ultra WildGuardMix OffsetBias JudgeBench RewardBench RewardBench 2

Benchmarks: RewardBench RewardBench 2 JudgeBench OffsetBias

Metrics: judge accuracy realized cost ratio accuracy-cost frontier

Baselines: All-Instruct All-Reasoning Random RouterBench-KNN RouteLLM-MF M-IRT ACER

Builds on: distributionally robust reinforcement learning ACER-style cost-constrained routing DPO

Closest papers by meaning

• Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.91)
• TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Tasks (ICLR 2026 · similarity 0.89)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget (ICML 2026 · similarity 0.88)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.88)
• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression (AAAI 2026 · similarity 0.88)
• CP-Router: An Uncertainty-Aware Router Between LLM and LRM (AAAI 2026 · similarity 0.88)