Research map / Time series / Foundation models for time series
AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoCast++, a zero-shot LLM-based retriever (relevance re-ranking plus recency weighting), LLM summarization, and a FiD T5 reader with human-alignment loss, improving forecasting accuracy on the Autocast benchmark.
Problem
Machine forecasting from news articles still lags human forecasters on the Autocast benchmark, largely because retrieved news contexts are noisy and not well aligned with question relevance and temporal dynamics.
Approach
Zero-shot LLM relevance scoring (binned expected rating) re-ranks BM25-retrieved articles, multiplied by a recency score derived from human forecaster accuracy changes; an LLM summarizes selected articles before a FiD T5 reader, with binning for numerical targets and an auxiliary loss aligning representations to human forecast accuracy.
Key results
- On the 0.2B model, Autocast++ reaches 43.8 MCQ accuracy vs 29.6 for FiD Static (about 48% relative gain) and 66.7 T/F accuracy vs 62.0.
- Autocast++ at 0.2B achieves 19.8 numerical absolute error vs 24.5 for FiD Static, while FiD Temporal (4.3B) attains the best numerical error of 19.5.
- Ablation shows zero-shot relevance re-ranking plus summarization lifts FiD Static MCQ from 29.6 to 42.1 and T/F from 62.0 to 65.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Autocast++ instead of FiD Static | T5 0.2B · Autocast test MCQ | accuracy: 29.6 -> 43.8 | Helped |
| Use Autocast++ instead of FiD Static | T5 0.2B · Autocast test T/F | accuracy: 62.0 -> 66.7 | Helped |
| Use Autocast++ instead of FiD Static | T5 0.2B · Autocast test Numerical | absolute error: 24.5 -> 19.8 | Helped |
| Add zero-shot relevance reranking and summarization (ablation) | FiD Static · Autocast test MCQ | accuracy: 29.6 -> 42.1 | Helped |
| Add zero-shot relevance reranking and summarization (ablation) | FiD Static · Autocast test T/F | accuracy: 62.0 -> 65.5 | Helped |
| Add human alignment loss (ablation) | FiD Static · Autocast test MCQ | accuracy: 29.6 -> 31.2 | Helped |
| Add numerical question binning (ablation) | FiD Static · Autocast test Numerical | absolute error: 24.5 -> 23.2 | Helped |
| Add zero-shot summarization (ablation) | FiD Static* · Autocast test MCQ | accuracy: 29.6 -> 42.3 | Helped |
| Add full retriever reranking, including recency (ablation) | FiD Static* · Autocast test MCQ | accuracy: 29.6 -> 42.4 | Helped |
| Remove numerical question binning (ablation) | Autocast++ · Autocast test Numerical | absolute error: 19.8 -> 21.0 | Hurt |
| Remove numerical question binning (ablation) | Autocast++ · Autocast test MCQ | accuracy: 43.8 -> 43.6 | No clear effect |
| Remove human alignment loss (ablation) | Autocast++ · Autocast test MCQ | accuracy: 43.8 -> 43.5 | Hurt |
Surprising findings
- Larger models give little benefit: the 2.8B Autocast++ is nearly identical to the 0.2B version on MCQ and numerical questions.
- The human-alignment loss helps the plain FiD Static but its contribution largely disappears once LLM re-ranking and summarization are added.
Limitations
- Relies on closed GPT-3.5-turbo API for relevance scoring and summarization, so results are not fully reproducible with open models.
- Numerical prediction remains weak; the FiD Temporal baseline is best there.
- Authors avoid newer LLMs to limit contamination, so comparisons to current models are absent.
Experimental setup
Datasets: Autocast Common Crawl news (2016-2022)
Benchmarks: Autocast
Metrics: accuracy absolute error
Baselines: UnifiedQA-v2 T5 FiD Static FiD Temporal
Builds on: Autocast Fusion-in-Decoder BM25 LoRA
Closest papers by meaning
- Approaching Human-Level Forecasting with Language Models (NeurIPS 2024 · similarity 0.85)
- Context is Key: A Benchmark for Forecasting with Essential Textual Information (ICML 2025 · similarity 0.85)
- A decoder-only foundation model for time-series forecasting (ICML 2024 · similarity 0.84)
- TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster (NeurIPS 2025 · similarity 0.84)
- From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection (NeurIPS 2024 · similarity 0.84)
- TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.