Research map / Reinforcement learning / Policy gradient and actor-critic

Risk-Controlled Post-Processing of Decision Policies

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Confounding-robust policy evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Post-processes a deterministic baseline policy with a score-based threshold switch to a fallback policy so that a violation-risk chance constraint holds, giving exact risk control with exact-safe fallbacks and O(log n/n) excess risk otherwise.

Problem

Choose a policy that agrees as much as possible with a deterministic baseline while keeping the probability of a loss-cutoff violation below a user-specified budget, using finite calibration data.

Approach

Population optimum is a threshold rule that switches from the baseline to an oracle fallback on contexts with largest conditional risk reduction; finite-sample version calibrates a threshold on a bumped empirical risk using a fitted score and fallback, with exact risk control when the fallback is exact-safe and rank-stability analysis otherwise.

Key results

• On the exact-safe COVID setting (c=7, RAC(0.05)), the post-processed policy reaches violation risk 0.02 at epsilon=0.02 with switch rate 0.06, versus pi_0 at 0.04.
• On LLM routing with Qwen3-4B -> Qwen3-32B thinking, at eps=0.25 post-processing uses 138.38 TFLOPs vs 157.96 for random mixing with matched violation risk 0.25.
• Synthetic experiment at c=7: algorithm reduces risk to about 0.175 with agreement 0.896, close to oracle risk 0.181 and agreement 0.914.

Surprising findings

• In the COVID non-monotone regime at c=2, the algorithm's mean realized loss (0.61) is slightly above the fallback's (0.54), despite the algorithm being designed to balance agreement and risk.

Limitations

• General fallback case requires rank-stability/drift assumptions that are not verified empirically (LLM routing).
• Results are for deterministic policies and finite action spaces; randomized policies and continuous actions are future work.
• Random-mixing baseline is empirical calibration without the conformal guarantee.

Experimental setup

Datasets: COVID-19 Radiography Database MMLU-Pro synthetic multiclass linear-softmax data

Benchmarks: COVID-19 Radiography Database MMLU-Pro synthetic multiclass task

Metrics: violation risk switch rate mean realized loss FLOPs agreement with baseline

Baselines: random mixing RAC baseline policy exact-safe fallback (Additional Testing)

Builds on: conformal risk control risk-averse calibration Inception-V3

Closest papers by meaning

• Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents (ICML 2025 · similarity 0.84)
• Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.83)
• Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity (NeurIPS 2025 · similarity 0.82)
• A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents (ICML 2025 · similarity 0.82)
• Predictive Performance Comparison of Decision Policies Under Confounding (ICML 2024 · similarity 0.82)
• SafeAR: Safe Algorithmic Recourse by Risk-Aware Policies (AAAI 2024 · similarity 0.82)