Research map / Time series / Long-term forecasting
The Effectiveness of Peer Prediction in Long-Term Forecasting
Mandal Debmalya, Radanović Goran, Parkes David
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add daily peer-prediction monetary payment instead of Brier scoring rule only (PP) | MTurk forecasting platform · user engagement (updates, returns, change) | engagement | Improved |
| add peer-prediction daily bonus on top of scoring rule (SR+PP) | scoring rule treatment · user engagement | engagement | Improved |
| add non-monetary peer-prediction quartile feedback (SR+PPRank) | scoring rule treatment · daily returns, all users | returns | Hurt |
| restrict to top-75 most updating users, SR+PPRank vs SR | SR+PPRank · engagement, top 75 users | updates and change | Improved |
| add rank feedback (SR+PPRank) vs SR | scoring rule · all 18 questions | mean final score: 0.725 -> 0.726 | No clear effect |
| add monetary peer prediction (SR+PP) vs SR | scoring rule · all 18 questions | mean final score: 0.725 -> 0.717 | No clear effect |
| add monetary peer prediction (SR+PP) vs SR on dynamic questions | scoring rule · dynamic questions | mean final score: 0.702 -> 0.690 | Hurt |
| add rank feedback (SR+PPRank) vs SR on hard questions | scoring rule · hard questions | mean final score: 0.641 -> 0.655 | No clear effect |
| add extremizing to top-k weighted mean aggregator (1.c to 1.d) (ablation) | SR forecasts · all questions | final accuracy: 0.819 -> 0.847 | Improved |
| add top-k users (k=75) to weighted mean aggregator (1.b to 1.c) (ablation) | SR forecasts · all questions | final accuracy: 0.810 -> 0.819 | Improved |
| use logit aggregation instead of mean (1.a to 2.a) (ablation) | SR forecasts · all questions | final accuracy: 0.804 -> 0.842 | Improved |
| add rank feedback (SR+PPRank) vs SR under 2.d on hard questions | scoring rule · hard questions, aggregator 2.d | final accuracy: 0.684 -> 0.778 | Improved |
Closest papers by meaning
- Improving Time Series Forecasting via Instance-aware Post-hoc Revision (NeurIPS 2025 · similarity 0.83)
- Approaching Human-Level Forecasting with Language Models (NeurIPS 2024 · similarity 0.82)
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities (ICLR 2025 · similarity 0.81)
- ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons (NeurIPS 2024 · similarity 0.81)
- Dirichlet-Guided Group Forecasting for Alleviating Over-smoothing in Time Series Forecasting (NeurIPS 2026 · similarity 0.80)
- Mixture of Online and Offline Experts for Non-Stationary Time Series (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
