Research map / Large language models / AI and society
Uncalibrated Models Can Improve Human-AI Collaboration
Kailas Vodrahalli, Tobias Gerstenberg, James Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| show optimized overconfident (uncalibrated) advice instead of calibrated advice | human-AI team (human behavior model simulation) · Simulation, 79% advice accuracy | final accuracy: +1.9% | Improved |
| show optimized modified advice to UK participants | human-AI team · Art and Cities, UK participants | final accuracy: +5.2% | Improved |
| show modified advice on tasks with higher advice accuracy (Art, Cities) vs lower (Sarcasm, Census) | modified advice · US human experiments | accuracy and confidence gain | Mixed |
| higher underlying advice accuracy (75% to 85%) with modified advice (ablation) | modified advice · Simulation | final accuracy: +0.7% -> +3.1% | Improved |
| use sigmoid-like transform instead of step function with low confidence (ablation) | modified advice · Art task | binned accuracy delta: 26.5% -> 28.5% | Improved |
| use sigmoid-like transform instead of high-confidence step function (ablation) | modified advice · Art task | binned accuracy delta: 29.1% -> 28.5% | No clear effect |
| use sigmoid-like transform instead of high-confidence step function (ablation) | modified advice · Art task | binned activation rate: 0.71 -> 0.77 | Improved |
| modified (sigmoid-like) advice vs unmodified advice | human-AI team · Art task | binned confidence delta: 0.19 -> 0.39 | Improved |
Closest papers by meaning
- Learning to Decide with AI Assistance under Human-Alignment (NeurIPS 2026 · similarity 0.88)
- Human-AI Teaming Through the Lens of Calibration (NeurIPS 2026 · similarity 0.88)
- Human Expertise in Algorithmic Prediction (NeurIPS 2024 · similarity 0.87)
- Conformal Prediction Sets Improve Human Decision Making (ICML 2024 · similarity 0.87)
- Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary (NeurIPS 2024 · similarity 0.87)
- A No Free Lunch Theorem for Human-AI Collaboration (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
