Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Teaching Humans When to Defer to a Classifier via Exemplars
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use exemplar-based teaching with feedback and neighborhood lessons instead of no teaching | Human-AI answering on HotpotQA · HotpotQA | Non-Defer F1: 57.6 $\pm$ 4.7 -> 67.6 $\pm$ 4.7 | Helped |
| use exemplar-based teaching instead of no teaching | Human-AI answering on HotpotQA · HotpotQA | Overall F1: 57.6 $\pm$ 3.4 -> 58.2 $\pm$ 3.4 | No clear effect |
| use exemplar-based teaching instead of no teaching | Human-AI answering on HotpotQA · HotpotQA | Defer F1: 57.8 $\pm$ 4.9 -> 50.7 $\pm$ 4.7 | Hurt |
| use exemplar-based teaching instead of LIME teaching | Human-AI answering on HotpotQA · HotpotQA | Overall F1: 52.9 $\pm$ 3.4 -> 58.2 $\pm$ 3.4 | Helped |
| require accurate lessons rather than inaccurate lessons (ablation) | Participants receiving the proposed teaching · HotpotQA | Overall F1: 53.5 $\pm$ 4.9 -> 62.8 $\pm$ 4.7 | Helped |
| teach examples from the test example's cluster rather than an unseen cluster (ablation) | Proposed teaching · HotpotQA ID vs OOD | Defer F1: 42.95 $\pm$ 17.2 -> 51.42 $\pm$ 4.9 | Mixed |
| increase teaching set to 4 examples using DOUBLE-GREEDY | CIFAR-10 human learner · CIFAR-10 test | Accuracy: 90.98 to 96.3 $\pm$ 0.1 | Helped |
| remove knowledge of the human predictor (ablation) | GREEDY-SELECT simulated learner · Synthetic experiments | accuracy | Hurt |
| remove knowledge of the prior rejector (ablation) | GREEDY-SELECT simulated learner · Synthetic experiments | accuracy | No clear effect |
| use noisy learned radii (ablation) | GREEDY-SELECT simulated learner · Synthetic experiments | accuracy | Mixed |
| allow up to 0.25 error in estimated cluster error probabilities (ablation) | DOUBLE-GREEDY in setting B · Synthetic experiments | performance: 0.25 | No clear effect |
| remove distractor paragraphs (ablation) | SAE-large · HotpotQA dev | F1: 79.62 -> 82.75 | Helped |
Closest papers by meaning
- Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary (NeurIPS 2024 · similarity 0.85)
- SKILL-MIX: a Flexible and Expandable Family of Evaluations for AI Models (ICLR 2024 · similarity 0.85)
- Cascaded Language Models for Cost-Effective Human–AI Decision-Making (NeurIPS 2025 · similarity 0.84)
- DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback (ICLR 2025 · similarity 0.84)
- AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.84)
- ExpeL: LLM Agents Are Experiential Learners (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.