Research map / Large language models / Reasoning and chain of thought

Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Casts LLM routing with costly value estimation as Pandora's Box with non-obligatory inspection, deriving closed-form VoI-based policies (Pandora's Router) and a decentralized posted-price bidding variant (Pandora's Bidder).

Problem

Routing queries to heterogeneous specialists requires estimating each specialist's expected return, but accurate value estimators are costly while cheap ones are noisy; the router must decide which costly estimates to compute.

Approach

Treat each specialist as a box in Pandora's Box with non-obligatory inspection; under a Gaussian signal model from cheap to costly estimates, reservation prices and value-of-information expressions are closed-form, deciding which expensive estimates to query. A decentralized variant lets specialists decide whether to refine self-assessments before accepting a posted price.

Key results

• On EmbedLLM averaged over query costs, Pandora's Router achieves total regret+cost 0.386 vs 0.389 for Margin-N_pr and 2.356 for g-always, with 0.075 inspection cost.
• On RAG averaged over costs, Pandora's Router achieves total 0.118 versus 0.128 for Margin-N_pr and 0.141 for g-only.
• On MATH, Pandora's Router achieves total 0.105, nearly identical to Margin-N_pr (0.105), and better than g-only (0.128).

Surprising findings

• With inaccurate competing prices (set from cheap f), the decentralized bidder increases its own surplus while reducing overall allocative efficiency.
• A non-Gaussian signal model improved distribution fit but did not improve routing success or inspection cost.

Limitations

• Gaussian signal model may not capture heavy tails or bounded/multimodal value distributions.
• Only two estimators (cheap f and costly g) are considered; the bidder's VoI is myopic and ignores multi-round strategic anticipation.
• Decentralized bidding against weak competitors can raise the strategic bidder's surplus at the expense of overall welfare.

Experimental setup

Datasets: MATH Omni-Math AIME HMMT PopQA Entity Questions Natural Questions BioASQ EmbedLLM GPQA GSM8K MMLU

Benchmarks: EmbedLLM MATH-based corpus (MATH, Omni-Math, AIME, HMMT) RAG factoid QA (PopQA, Entity Questions, Natural Questions, BioASQ)

Metrics: routing regret plus inspection cost specialist surplus allocative efficiency regret monetary cost

Baselines: f-only g-always Top-2 Coin Flip Random-N_pr Margin-N_pr

Builds on: Pandora's Box problem non-obligatory inspection (Pandora-NI) information value theory EmbedLLM

Closest papers by meaning

• Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning (ICML 2026 · similarity 0.86)
• Causal LLM Routing: End-to-End Regret Minimization from Observational Data (NeurIPS 2025 · similarity 0.85)
• Cost-of-Pass: An Economic Framework for Evaluating Language Models (ICLR 2026 · similarity 0.85)
• The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs (ICML 2026 · similarity 0.84)
• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
• Cost-aware Bayesian Optimization via the Pandora's Box Gittins Index (NeurIPS 2024 · similarity 0.84)