Research map / Large language models / Language, safety and interpretability
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Conformal Arbitrage, a black-box post-hoc router that uses a single conformal-risk-control-calibrated score-gap threshold to defer between a Primary LLM and a Guardian LLM (or human) with finite-sample guarantees on a guardrail loss.
Problem
Balancing competing LLM objectives (helpfulness vs harmlessness, cost vs accuracy) at deployment time when models are API-only and weights cannot be changed, while guaranteeing a user-specified risk budget on a guardrail metric.
Approach
Calibrate one scalar relaxation threshold on the Primary's scores with conformal risk control; if the Primary's candidate set within that threshold is a singleton, return it, otherwise let the Guardian pick from the relaxed candidate set. Needs only scores, no logits or weights.
Key results
- On TruthfulQA, CA at alpha=0.05 reaches 0.806 accuracy vs 0.833 Guardian and 0.559 Primary, at about 84% of Guardian cost (0.521 vs 0.620 $/1000).
- On PKU-SafeRLHF, CA at alpha=0.30 halves the Primary's severity loss (0.245 vs 0.676) while retaining 63% of its helpfulness, with Guardian invoked on about 72% of queries.
- On MMLU at alpha=0.10, CA recovers 91% of Guardian accuracy (0.753 vs 0.828) at 61% of its cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use calibrated Conformal Arbitrage instead of cost-matched random routing | Primary/Guardian routing · TruthfulQA multiple-choice | accuracy: CA beats the cost-matched random router at every risk level except α=0.25 | Helped |
| Tighten the risk budget from α=0.25 to α=0.05 | Conformal Arbitrage · TruthfulQA multiple-choice | accuracy: 0.62 to 0.81 | Helped |
| Use Conformal Arbitrage rather than randomized routing | Primary/Guardian routing · PKU-SafeRLHF | Empirical Human Alignment | Mixed |
| Increase calibration size from 300 to 500 examples (ablation) | Conformal Arbitrage · TruthfulQA multiple-choice | accuracy: at most 1-2 percentage points | No clear effect |
| Use the Guardian's raw confidence scores instead of 0/1 binarized scores (ablation) | Conformal Arbitrage · TruthfulQA multiple-choice | accuracy: roughly +1-2% when α≤0.10; about 0.5%-1% lower when α≥0.20 | Mixed |
| Let the Guardian choose from the full action set instead of the Primary's relaxed subset (ablation) | Conformal Arbitrage · TruthfulQA multiple-choice | accuracy: roughly 3-6 percentage points | Helped |
| Replace the gpt-4.1-nano Primary with the stronger gpt-4.1-mini (ablation) | Conformal Arbitrage · TruthfulQA multiple-choice | accuracy: at α=0.05 it gains +2 pp over cost-matched random routing; at α=0.025 it matches Guardian accuracy | Helped |
| Use Conformal Arbitrage at α=0.10 | Primary/Guardian routing · MMLU | accuracy: recovers 91% of the Guardian’s accuracy while spending 61% of its cost | Helped |
| Tighten the risk budget from α=0.60 to α=0.10 | Conformal Arbitrage · PKU-SafeRLHF | Safety-violation loss: 0.571 \pm 0.070 to 0.080 \pm 0.022 | Helped |
Surprising findings
- Unrestricted routing (Guardian sees full action set) raises accuracy by 3-6 points but cost overshoots the restricted variant by about 7-10%.
- On MMLU the advantage over cost-matched random routing is smaller and degrades at both extremes of alpha; at alpha=0.05 CA is slightly below random (delta -0.005).
Limitations
- Evaluated only on multiple-choice tasks; free-text generation would need bespoke loss functions.
- Single-step two-model router; deeper cascades not studied.
- Guardian binarization on TruthfulQA ties accuracy loss to Guardian accuracy; the paper forgoes task-specific cost-accuracy optimization and does not claim to beat specialized cascades.
Experimental setup
Datasets: TruthfulQA MMLU PKU-SafeRLHF
Benchmarks: TruthfulQA MMLU PKU-SafeRLHF
Metrics: accuracy cost per 1000 examples severity-loss guardian usage rate delta over random router
Baselines: Primary-only Guardian-only cost-matched random router CSE (Cascaded Selective Evaluation)
Builds on: conformal risk control conformal prediction Chow reject-option classifier Cascaded Selective Evaluation
Closest papers by meaning
- Conformal Thinking: Risk Control for Reasoning on a Compute Budget (ICML 2026 · similarity 0.87)
- Language Models Can Predict Their Own Behavior (NeurIPS 2025 · similarity 0.87)
- Cost-of-Pass: An Economic Framework for Evaluating Language Models (ICLR 2026 · similarity 0.87)
- Conformal Tail Risk Control for Large Language Model Alignment (ICML 2025 · similarity 0.87)
- Inference-Time Reward Hacking in Large Language Models (NeurIPS 2025 · similarity 0.87)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.