Research map / Learning theory and algorithms / Prediction and decision losses
Proper Proxy Scoring Rules
Jens Witkowski, Pavel Atanasov, Lyle Ungar, Andreas Krause
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use proxy scores instead of proper scores for the selection set when it has only 1-4 questions | quadratic scoring rule · Good Judgment Project | agreement with validation winner | Improved |
| use proxy scores instead of proper scores for the selection set with 30 questions | quadratic scoring rule · Good Judgment Project | probability of calling correct winner: 0.73 vs 0.82 (about 89%) | Hurt |
| score in sample with proxy rather than out of sample with proper scores | quadratic scoring rule · Good Judgment Project, 30 questions | agreement with validation winner: 0.71 vs 0.71 | No clear effect |
Closest papers by meaning
- Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions (NeurIPS 2025 · similarity 0.85)
- Hedging and Approximate Truthfulness in Traditional Forecasting Competitions (AAAI 2025 · similarity 0.84)
- Consistency Checks for Language Model Forecasters (ICLR 2025 · similarity 0.83)
- Introducing FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark (NeurIPS 2025 · similarity 0.81)
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities (ICLR 2025 · similarity 0.81)
- Forecasting Competitions with Correlated Events (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
