Research map / Learning theory and algorithms / Submodular and game algorithms
Hedging and Approximate Truthfulness in Traditional Forecasting Competitions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives the first strategic analysis of the traditional "Simple Max" forecasting competition mechanism over multiple events, showing long-run truthfulness folklore fails (best forecaster hedges) but two near-equal forecasters with sufficient uncertainty are approximately truthful.
Problem
Whether the traditional forecasting competition mechanism (Simple Max, highest total quadratic score wins) remains approximately truthful as the number of events grows, a folklore claim lacking formal analysis.
Approach
Shows via a geometric argument on a p-biased coin model that a far-ahead informed forecaster strictly benefits from hedging toward the prior for any number of events, and uses Edgeworth expansions to show that two forecasters with sufficient uncertainty over opponent reports and outcomes are approximately (l-infinity) truthful.
Key results
- For p=0.9 and m>=32, deviating by 0.2 in every coordinate strictly beats any 0.04 l2-approximately truthful strategy, showing a counterexample to long-run truthfulness.
- Two forecasters with sufficient uncertainty (Condition on variance, smoothness, and competitiveness) are gamma-l-infinity approximately truthful with gamma = O(sigma_i^{-1/2}), where sigma_i is the score-difference standard deviation.
- Exact equilibria for m=1 and n=2 give win probabilities 3/4 - p/2 for the informed forecaster and 1/4 + p/2 for the uninformed one.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Hedge toward the prior by reporting r* = (p*, ..., p*), where p* = (1/2 + p)/2, instead of an approximately truthful report | Informed forecaster under Simple Max, against approximately truthful opponents · Independent p-biased binary events, under Condition 1 | win probability | Helped |
| Use Simple Max when both forecasters have sufficient uncertainty about the opponent's reports, with the stated score-difference conditions | Two-forecaster Simple Max · Forecasting competitions satisfying Conditions 1 and 2 | maximum report deviation from belief (ℓ∞): γ = O (σ_i^{-1/2}) | Helped |
| Have the informed forecaster report 0 and uninformed forecasters randomize over the endpoints rather than report their beliefs | Simple Max with n forecasters and one event · m=1 | informed forecaster win probability: 2/n - 1/2^{n-1}[1/n - p/(n+1)] | Mixed |
Surprising findings
- Even with arbitrarily many events, the best forecaster can strictly benefit from hedging toward the prior, contrary to long-run truthfulness folklore.
- Bad forecasters extremize while good forecasters hedge, and the hedging incentive persists as the number of events grows.
Limitations
- Hedging counterexample analyzed only for m=1,2 exactly; general equilibrium for m>2 unknown.
- Approximate truthfulness result requires two forecasters and conditions that may fail as the number of events grows; extension to n>2 is only conjectured.
- Analysis uses the quadratic score and a Bayesian p-biased coin model.
Experimental setup
Metrics: win probability l-infinity approximate truthfulness
Builds on: Soft Max mechanism
Closest papers by meaning
- Forecasting Competitions with Correlated Events (AAAI 2025 · similarity 0.87)
- Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions (NeurIPS 2025 · similarity 0.85)
- Truthfulness of Calibration Measures (NeurIPS 2024 · similarity 0.84)
- Online Learning in Betting Markets: Profit versus Prediction (ICML 2024 · similarity 0.84)
- Consistency Checks for Language Model Forecasters (ICLR 2025 · similarity 0.83)
- Carrot and Stick: Eliciting Comparison Data and Beyond (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.