AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Curating the Future: A Scalable Recipe for Training Open-Ended Forecasters

Nikhil Chandak, Shashwat Goel, Ameya Prabhu, Moritz Hardt, Jonas Geiping

ICML 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add leakage-removal filtering to generated questions (ablation)RL on OpenForesight, Qwen3-8B · Validation setaccuracy: worsens without removal; improves after leakage removalImproved
add all filtering stages on top of leakage removal (ablation)RL on OpenForesight, Qwen3-8B · Validation setaccuracy, Brier score: 3x less data and stepsImproved
add accuracy term to Brier-only reward (ablation)GRPO reward · Guardian validation setaccuracy, Brier score: 'Unknown' answers ~40% -> ~4%Improved
add Brier term to accuracy-only reward (ablation)GRPO reward · Guardian validation setBrier score: negative Brier scores, worse than constant 0Improved
add retrieval of 5 news chunks (ablation)forecasters across model families · Guardian validation setaccuracy: 9-18% accuracy gainImproved
retrieve more than 5 chunks (ablation)Qwen3-8B, gpt-oss-120b · Guardian validation setaccuracy: plateau after five chunksNo clear effect
train in temporal order instead of shuffled (ablation)RL training · Validation setaccuracyNo clear effect
train on free-form questions instead of binary-only (ablation)Qwen3-8B RL · Free-form validation setfree-form Brier score: 0.004 -> 0.086Improved
train on free-form only versus mix of binary and free-form (ablation)Qwen3-8B RL · Free-form validation setfree-form Brier score: 0.086 vs 0.065Mixed
add binary questions to free-form training data (ablation)Qwen3-8B RL · Metaculus binary questionsaccuracy, binary Brier: binary-only and mix improve; free-form only does notImproved
RL training on OpenForesightQwen3-8B · Guardian validation set, world newsAvg@3 accuracy (%): 21.7 -> 33.3Improved
RL training on OpenForesightQwen3-8B · Guardian validation set, sportAvg@3 accuracy (%): 43.2 -> 43.2No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy