Research map / Learning theory and algorithms / Submodular and game algorithms
Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing
Nihar B. Shah, Dengyong Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| skip option with threshold T=0.75 instead of attempting every question | majority-vote labeling in synthetic simulation · Synthetic simulations (five confidence distributions) | fraction of questions labeled incorrectly | Improved |
| skip-based setting under hammer-spammer distribution versus other distributions | skip-based mechanism · Synthetic simulations (hammer-spammer) | fraction of questions labeled incorrectly | Improved |
| add noise to worker confidence and count ties as error 0.4 (ablation) | skip-based setting · Synthetic simulations (perturbed) | fraction of questions labeled incorrectly | No clear effect |
| use skip-based multiplicative mechanism instead of fixed payment per correct answer | MTurk raw labels · Amazon Mechanical Turk (nine tasks) | error among attempted questions | Improved |
| use skip-based mechanism instead of baseline | MTurk labels · Amazon Mechanical Turk (nine tasks) | average payment to worker | No clear effect |
| use skip-based mechanism, then aggregate by majority vote over 3 to 11 workers | MTurk multiple-choice tasks · Amazon Mechanical Turk (five multiple-choice tasks, aggregated) | aggregate labeling error | Improved |
Closest papers by meaning
- No Oops, You Won’t Do It Again: Mechanisms for Self-correction in Crowdsourcing (ICML 2016 · similarity 0.91)
- Sequential Peer Prediction: Learning to Elicit Effort using Posted Prices (AAAI 2017 · similarity 0.88)
- Peer Neighborhood Mechanisms: A Framework for Mechanism Generalization (AAAI 2024 · similarity 0.88)
- Evaluating LLM-contaminated Crowdsourcing Data Without Ground Truth (NeurIPS 2025 · similarity 0.85)
- Position: Scarce Resource Allocations That Rely On Machine Learning Should Be Randomized (ICML 2024 · similarity 0.83)
- Carrot and Stick: Eliciting Comparison Data and Beyond (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
