Research map / Large language models / AI and society
Human Expertise in Algorithmic Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes algorithmic indistinguishability via multicalibrated partitions to selectively incorporate human expert judgments into algorithmic predictions, showing provable improvement over any predictor in a class and that radiologists beat all leaderboard algorithms on ~30% of chest X-ray patients.
Problem
When and how human expert judgment can improve the predictions of any learning algorithm, given that experts may use side information not encoded in algorithm inputs, in high-stakes prediction tasks such as medical diagnosis.
Approach
Define α-indistinguishable subsets (no predictor in a class has significant covariance with the outcome within the subset) and build a multicalibrated partition; within each subset, regress the outcome on the human prediction, which provably beats any predictor in the class and yields a test for informative experts.
Key results
- Radiologists are statistically indistinguishable from the eight leaderboard algorithms overall on atelectasis (MCC comparison with bootstrap CIs).
- In an indistinguishable subset containing 29.6% of patients (n=148), radiologists achieve a better MCC than every algorithm, while all algorithms predict positive for every patient there.
- On the visual prediction task, the predictor h achieves 72.2% out-of-sample accuracy versus 67.3% for the best human cohort, yet human predictions remain correlated with the outcome within every nonempty level set of h.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use radiologist predictions instead of the leaderboard algorithms on indistinguishable subset 0 | Atelectasis classification · Atelectasis, indistinguishable subset 0 | MCC | Helped |
| Use radiologist predictions instead of algorithm predictions on the full evaluation set | Atelectasis classification · Atelectasis, overall | MCC | No clear effect |
| Use the boosted predictor h instead of the best-performing human cohort | Escape the Room photo prediction · Escape the Room photo prediction, out of sample | accuracy: 67.3% -> 72.2% | Helped |
| Use human predictions within the nonempty level sets of h | Escape the Room photo prediction · Nonempty level sets of h | predictive signal | Helped |
| Use radiologist predictions on the atelectasis subset where all benchmark algorithms predict positive | Atelectasis classification · Indistinguishable subset 0 | TNR: 0% -> 42.9% | Helped |
Surprising findings
- Human predictions can be predictive within every level set of an algorithm even when the algorithm outperforms humans overall, illustrated via Simpson's-paradox-like structure.
- All eight algorithms predict positive for every patient in the identified subset, yet radiologists identify many true negatives there.
Limitations
- Focuses on minimizing a fixed loss over a stationary distribution; does not handle multidimensional preferences such as fairness or robustness.
- Does not address performative settings where predictions influence outcomes.
- Learning multicalibrated partitions can be challenging for rich function classes; confidence intervals for subset results are not strictly valid because subsets are chosen conditional on predictions.
Experimental setup
Datasets: CheXpert-style chest radiograph dataset (224,316 radiographs, 65,240 patients) 500-radiograph radiologist-annotated holdout Escape the Room photo dataset (visual prediction task)
Benchmarks: atelectasis chest X-ray classification holdout (500 radiographs) Escape the Room visual prediction task
Metrics: Matthews Correlation Coefficient true positive rate true negative rate squared error
Baselines: eight leaderboard X-ray classifiers five off-the-shelf visual prediction algorithms shallow regression trees (depth<=5)
Builds on: omnipredictors multicalibration boosting for multicalibration (Gopalan et al.) outcome indistinguishability swap agnostic learning
Closest papers by meaning
- Human-AI Collaborative Uncertainty Quantification (ICML 2026 · similarity 0.89)
- Towards Human-AI Complementarity with Prediction Sets (NeurIPS 2024 · similarity 0.87)
- A No Free Lunch Theorem for Human-AI Collaboration (AAAI 2025 · similarity 0.86)
- Learning to Decide with AI Assistance under Human-Alignment (NeurIPS 2026 · similarity 0.86)
- Human and AI Perceptual Differences in Image Classification Errors (AAAI 2025 · similarity 0.85)
- Conformal Prediction Sets Improve Human Decision Making (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.