Research map / Label-efficient and robust learning / Recognition and long-tail
Designing Decision Support Systems using Counterfactual Prediction Sets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits conformal prediction sets as decision support for human experts, proposing counterfactual successive elimination and counterfactual UCB1 that exploit nested prediction sets and a counterfactual monotonicity assumption to find the accuracy-optimal conformal predictor without an expert model, validated in a large human subject study on ImageNet16H.
Problem
Choosing the conformal predictor (coverage level) that maximizes the average accuracy of real human experts who must pick a label from a prediction set, without assuming a stylized expert model.
Approach
Leverages the nested structure of conformal prediction sets plus a counterfactual monotonicity assumption (if an expert succeeds/fails on a set containing the true label, they would succeed/fail on a smaller/larger set also containing it) to infer rewards for many arms from one pull, yielding exponentially better regret than vanilla bandits in successive elimination.
Key results
- Counterfactual SE and counterfactual UCB1 achieve clear regret reductions over vanilla SE and UCB1 over 1,080 rounds averaged across 30 realizations.
- Under the strict implementation, experts reach higher success probability than under the lenient implementation across competitive alpha values, with human experts alone at 0.760 versus classifier accuracy 0.848.
- Sensitivity analysis: with monotonicity violations for all images, the alpha found by counterfactual UCB1 is only 3.8% lower in success probability than the optimum.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Limit experts to labels in the prediction set rather than allowing an outside-set choice | Prediction-set decision support system · ImageNet16H-PS | expert accuracy | Helped |
| Use counterfactual rewards with successive elimination | Successive elimination · ImageNet16H-PS, 1{,}080-step regret evaluation | expected regret | Helped |
| Use counterfactual rewards with UCB1 | UCB1 · ImageNet16H-PS, 1{,}080-step regret evaluation | expected regret | Helped |
| Use counterfactual UCB1 instead of Counterfactual SE | Counterfactual bandit algorithms · ImageNet16H-PS, regret evaluation | expected regret | Helped |
| Use Counterfactual SE instead of the algorithm by Straitouri et al. | Strict prediction-set decision support system · ImageNet16H-PS | expert accuracy | Helped |
| Use smaller prediction sets that still contain the true label | Strict prediction-set decision support system · ImageNet16H-PS, images not too easy | expert accuracy | Helped |
| Increase counterfactual monotonicity violations across all images | Counterfactual UCB1 · ImageNet16H-PS, all images permuted | accuracy relative to optimal α: 3.8\% lower | Hurt |
Surprising findings
- Strictly limiting expert agency (strict prediction sets) yields higher expert accuracy than a lenient system that allows experts to choose labels outside the set.
- Counterfactual UCB1 empirically achieves lower regret than Counterfactual SE, though no formal guarantee is proven for it.
Limitations
- Counterfactual monotonicity is a level-three counterfactual assumption that cannot be validated experimentally; only interventional monotonicity is checked.
- Study covers one classification task on a single natural-image benchmark with non-expert crowdworkers rather than domain experts.
- Assumes i.i.d. data and noiseless ground-truth labels in the calibration set; theory covers only Counterfactual SE.
Experimental setup
Datasets: ImageNet16H
Benchmarks: ImageNet16H-PS
Metrics: expected regret empirical success probability of expert predictions
Baselines: vanilla successive elimination vanilla UCB1 Straitouri et al. (stylized expert model MNL-based algorithm) lenient implementation of prediction sets
Builds on: successive elimination UCB1 conformal prediction structural causal model
Closest papers by meaning
- Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Sets (NeurIPS 2024 · similarity 0.91)
- Towards Human-AI Complementarity with Prediction Sets (NeurIPS 2024 · similarity 0.90)
- Conformal Prediction Sets Improve Human Decision Making (ICML 2024 · similarity 0.87)
- Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents (ICML 2025 · similarity 0.86)
- Efficient Online Set-valued Classification with Bandit Feedback (ICML 2024 · similarity 0.85)
- Counterfactual Online Conformal Prediction Under Adaptive Logging (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.