Research map / Learning theory and algorithms / Sample complexity
Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence
Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin J. Wainwright
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Circle task | error: 17% -> 6% | Improved |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Age task | error: 17% -> 13% | Improved |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Tagline task | error: 42% -> 44% | No clear effect |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Spelling task | error: 42% -> 40% | Improved |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Distance task | error: 20% -> 17% | Improved |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Audio task | error: 29% -> 20% | Improved |
| elicit ordinal comparisons directly instead of converting cardinal ratings | MTurk crowdsourcing · Relevance task | error: 35% -> 31% | Improved |
| elicit ordinal comparisons instead of cardinal ratings | MTurk crowdsourcing · Spelling task | median completion time: 525s -> 316s | Improved |
| elicit ordinal comparisons instead of cardinal ratings | MTurk crowdsourcing · Audio task | median completion time: 134s -> 66s | Improved |
| elicit ordinal comparisons instead of cardinal ratings | MTurk crowdsourcing · Tagline task | median completion time: 342s -> 251s | Improved |
| use ordinal Thurstone MLE rather than cardinal MLE with multiple workers | MTurk crowdsourcing · Distance task | normalized squared error: 0.330 -> 0.168 | Improved |
| use ordinal Thurstone MLE rather than cardinal MLE with multiple workers | MTurk crowdsourcing · Spelling task | normalized squared error: 0.350 -> 0.358 | No clear effect |
Closest papers by meaning
- Generalized Results for the Existence and Consistency of the MLE in the Bradley-Terry-Luce Model (ICML 2022 · similarity 0.89)
- A New and Flexible Approach to the Analysis of Paired Comparison Data (JMLR 2018 · similarity 0.89)
- Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions (JMLR 2022 · similarity 0.89)
- Pairwise Comparisons without Stochastic Transitivity: Model, Theory and Applications (JMLR 2026 · similarity 0.89)
- Aggregating Quantitative Relative Judgments: From Social Choice to Ranking Prediction (NeurIPS 2024 · similarity 0.85)
- Hypothesis Testing for Generalized Thurstone Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
