Research map / Probabilistic and scientific modeling / Molecules and proteins
Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that conditioning performative forecasts on separating covariates makes the target forecast-invariant, proves classical proper scoring rules fail in general performative settings, and proposes utility-based and unbiased-divergence scoring rules that elicit correct forecasts and yield performatively stable parameter estimates.
Problem
Performative forecasts influence the outcomes they predict, so correct forecasts may not exist and classical proper scoring rules and estimators may fail to elicit or estimate correct forecasts.
Approach
Condition forecasts on covariates that separate forecast from outcome (forecast-invariance via d-separation), show classical proper scores are generally not proper under performativity, and fix this with a utility score plus scaled strictly proper score in decision settings, or by scoring with unbiased estimates of the performative divergence.
Key results
- Theorem shows the existence of a correct conditional forecast is equivalent to d-separation Y ⊥ F | A (forecast-invariance).
- Theorem shows no non-constant classical proper scoring rule is observationally proper for performative settings with forecast-invariant targets under the stated mechanisms.
- Unbiased estimators of the performative divergence (Brier and energy score) are proven unbiased and therefore proper under Corollary conditions.
Surprising findings
- Classical proper scoring rules, even when strictly proper, can be improper under performativity: forecasters can benefit from misreporting for unobserved actions.
- Correct forecasts can be incentive-incompatible with the agent's utility, so being correct can be harmful.
Limitations
- Forecast-invariance assumption may be hard to motivate in practice.
- Assumes a discrete action space A; continuous case left open.
- Results are for probabilistic forecasts; point forecasts only conjectured.
Experimental setup
Metrics: expected score divergence bias of estimators
Baselines: classical proper scoring rules (Brier, log-score, energy score) utility score inverse probability weighting score
Builds on: performative prediction (Perdomo et al. 2020) Bregman score Bayes act utility score inverse probability weighting
Closest papers by meaning
- Truthfulness of Calibration Measures (NeurIPS 2024 · similarity 0.85)
- Statistical Inference under Performativity (NeurIPS 2025 · similarity 0.85)
- Hedging and Approximate Truthfulness in Traditional Forecasting Competitions (AAAI 2025 · similarity 0.85)
- Plug-in Performative Optimization (ICML 2024 · similarity 0.84)
- The Stability of Online Algorithms in Performative Prediction (NeurIPS 2026 · similarity 0.84)
- ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.