Research map / Learning theory and algorithms / Prediction and decision losses

Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Probabilistic calibration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Conformal Selective Acting (CSA), a per-round anytime-valid wrapper using per-threshold e-processes on the selective risk increment to certify a deployment threshold for RLVR-trained local LLMs under a per-deployment error budget.

Problem

Certifying selective (released-output) verifier-failure risk at every wall-clock round on a single non-poolable deployment stream where an RLVR-trained specialist LLM is updated online, violating exchangeability and long-run-average assumptions of prior conformal wrappers.

Approach

Maintain a Ville-type e-process per candidate score threshold on the gated selective increment A_t((1-V_t)-alpha), with a Bonferroni grid and a max-certified-threshold deployment rule, so that the released threshold is anytime-pathwise valid under predictable updates and isotonic-calibrated monotone risk.

Key results

• CSA achieves 0/480 pathwise violations across eight specialist benchmarks while remaining non-refusing, whereas five online baselines violate 10/10 per cell at pivotal alpha.
• Across five live RLVR cells (100 replications), CSA has PathV=0/100 with action rate at least 50% on every cell; offline conformal methods are pathwise valid but have 1.0-19x lower action rates.
• Across sixteen adversarial distribution-shift cells (160 streams), CSA has 0/160 pathwise violations and the lowest mean risk (3.2%).

Surprising findings

• Offline-conformal baselines such as CRC refuse (zero action rate) on most benchmarks at the pivotal alpha, so pathwise validity is often achieved only by abstaining.

Limitations

• Guarantee is with respect to a deterministic verifier; imperfect or LLM-judge verifiers are out of scope.
• Requires one predictable update step between rounds; mid-batch updates need the multi-epoch variant.
• Relies on isotonic-calibrated monotone risk and bounded within-epoch frontier drift for no-false-certification.

Experimental setup

Datasets: MedQA PubMedQA TAT-QA MedNLI GSM8K HEAD-QA ARC-Challenge CaseHOLD MATH

Benchmarks: MedQA PubMedQA TAT-QA MedNLI GSM8K HEAD-QA ARC-Challenge CaseHOLD MATH

Metrics: pathwise violations (PathV) selective risk action rate refusal rate

Baselines: CRC LTT ConfFact NEX-Conf ACI SAOCP A-RCPS Fixed-Threshold Naive-Tuning Always-Act

Builds on: A-RCPS RCPS LTT Ville's inequality Expert Iteration conformal risk control

Closest papers by meaning

• Calibration Is Not Control: Intervention Advantage for LLM-Agent Oversight (NeurIPS 2026 · similarity 0.86)
• A Provable Approach for End-to-End Safe Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• Test-time Risk Adaptation with Mixture of Agents (NeurIPS 2026 · similarity 0.85)
• Tricks or Traps? A Deep Dive into RL for LLM Reasoning (ICLR 2026 · similarity 0.84)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data (ICLR 2026 · similarity 0.84)
• On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation (NeurIPS 2026 · similarity 0.84)