Research map / Learning theory and algorithms / Prediction and decision losses
When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes a human-algorithm collaboration where the algorithm presents k of n items to a human who picks one, showing under Mallows and RUM noise that joint performance can beat either agent alone (k in [2,n-1]) unless the human anchors on the algorithm's ordering, which always hurts.
Problem
When does a human-algorithm system in which the algorithm narrows n items to k presented items, from which the human picks the final one, outperform either agent alone at finding the best item, given noisy and possibly anchored rankings?
Approach
Uses a bijection between "good" and "bad" events (best-item mapping) and Kendall-tau inversion counting under the Mallows model to prove complementarity for k=2 with equal accuracy and unanchored rankings; shows anchoring reverses this, and checks the pattern numerically under Random Utility Models.
Key results
- For the Mallows model with n=3, k=2 and equal accuracy, the joint system strictly outperforms either agent alone (theorem proven via best-item mapping, for any n>2).
- With full anchoring (w_a=1), the joint system always has strictly worse accuracy than the algorithm alone for any k<n and any accuracy levels.
- Under the RUM with n=10, k=2, the complementarity region includes the equal-accuracy diagonal and extends further when the human is more accurate than when the algorithm is more accurate.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Present exactly 2 items in the unanchored, equal-accuracy setting | Mallows human-algorithm system · Unanchored; equal human and algorithm accuracy | probability of picking the best item | Helped |
| Let a more accurate human collaborate with a less accurate algorithm in the stated range | Mallows human-algorithm system · n=3, k=2 | probability of picking the best item: φ_a ≥ max(φ_h/1.3, φ_h - 0.3) | Helped |
| Let a more accurate algorithm collaborate with a less accurate human within the identified narrow range | Mallows human-algorithm system · n=3, k=2 | probability of picking the best item: φ_a ∈ [φ_h, φ_h · (1 + 0.01)] | Helped |
| Increase algorithm accuracy beyond the stated threshold when the algorithm is more accurate | Mallows human-algorithm system · n=3, k=2; φ_h≥1 | probability of picking the best item: φ_a ≥ φ_h + 0.15 | Hurt |
| Place the more accurate agent in the human role rather than the algorithm role | Mallows human-algorithm system · n=3, k=2; differing accuracies | probability of picking the best item | Helped |
| Fully anchor the human on the algorithm's realized ordering | Mallows human-algorithm system · Anchored; any k<n | probability of picking the best item | Hurt |
| Use partial rather than no anchoring | Mallows human-algorithm system · n=5; equal accuracy; intermediate anchoring strengths | complementarity: w_a ∈ {0, 0.25, 0.5, 0.75, 1} | Mixed |
| Use partial rather than no anchoring | Random Utility human-algorithm system · n=5; equal accuracy; intermediate anchoring strengths | complementarity | Mixed |
| Collaborate with equal accuracy and present 2 items | Random Utility human-algorithm system · Normal RUM; n=10; unanchored | complementarity | Helped |
| Make human and algorithm accuracy differ, with the human more accurate | Random Utility human-algorithm system · Normal RUM; n=10; unanchored; k=2 | complementarity | Helped |
| Increase Mallows accuracy from φ=1 to φ=1.3 | Human or algorithm acting alone · Mallows; n=3 | probability of recovering the correct permutation: 48% ... 57% | Helped |
Surprising findings
- Given a uniform distribution over permutations, good and bad events are equally likely, so complementarity must be driven by non-uniform permutation probabilities.
- Complementarity is asymmetric: a more accurate human benefits from a less accurate algorithm over a wider range than a more accurate algorithm benefits from a less accurate human.
Limitations
- Theoretical results are mostly for Mallows model with n=3, k=2; RUM results are numerical only.
- Models only the algorithm presenting a subset to a bandwidth-limited human; other collaboration protocols (voting, sequential inspection) are not analyzed.
- Human and algorithm accuracy are modeled as single parameters without empirical human data.
Experimental setup
Metrics: probability of selecting best item (accuracy) complementarity region
Builds on: Mallows model Random Utility Model complementarity (Bansal et al.)
Closest papers by meaning
- Learning to Decide with AI Assistance under Human-Alignment (NeurIPS 2026 · similarity 0.84)
- A No Free Lunch Theorem for Human-AI Collaboration (AAAI 2025 · similarity 0.83)
- Human Expertise in Algorithmic Prediction (NeurIPS 2024 · similarity 0.83)
- Model Alignment as Prospect Theoretic Optimization (ICML 2024 · similarity 0.83)
- Accuracy vs. Accuracy: Computational Tradeoffs Between Classification Rates and Utility (NeurIPS 2026 · similarity 0.83)
- The Hidden Cost of Waiting for Accurate Predictions (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.