Research map / Time series / Foundation models for time series
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
Md Atik Ahamed, Mihir Parmar, Palash Goyal, Yiwen Song, Long T. Le, Qiang Cheng, Chun-Liang Li, Hamid Palangi, Jinsung Yoon, Tomas Pfister
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prompt LLMs with verified reasoning traces instead of direct numerical prediction (ablation) | LLM forecasters · TFRBench (avg. over datasets) | forecasting success rate / accuracy (%): ~40.2% -> 56.6% | Improved |
| add reasoning-guided forecasting on Solar (ablation) | LLM forecaster · Solar | success rate (MASE<1) (%): 30.7% -> 70.4% | Improved |
| add reasoning-guided forecasting on Hierarchical Sales (ablation) | LLM forecaster · Hierarchical Sales | success rate (MASE<1) (%): 54.7% -> 78.5% | Improved |
| add CoT reasoning in pattern-rich domain Solar (ablation) | Gemini-3-Pro · Solar | MASE: 1.037 -> 0.834 | Improved |
| add CoT reasoning in Car Parts (ablation) | Gemini-3-Pro · Car Parts | MASE: 0.821 -> 0.672 | Improved |
| add CoT reasoning in stochastic Finance domain (ablation) | Gemini-3-Pro · Amazon Pricing | MASE: 0.715 -> 1.114 | Hurt |
| add event forecasting and external search to reasoning (Energy) (ablation) | Gemini-2.5-Pro · Electricity | MASE: 2.853 -> 1.327 | Improved |
| add event forecasting to reasoning (Transportation) (ablation) | Claude-Sonnet-4.5 · Traffic | MASE: 1.458 -> 1.285 | Improved |
| add event-aware reasoning in Web/CloudOps (ablation) | Claude-Sonnet-4.5 · Bitbrains Fast Storage | MASE: 0.938 -> 5.469 | Hurt |
| add event forecasting to reasoning (ablation) | Gemini-2.5-Flash · Solar | MASE: 1.168 -> 0.896 | Improved |
| use larger model Gemini-3-Pro instead of Gemini-2.5-Pro for reasoning | LLM reasoning · Solar | LLM judge average score: 2.977 -> 3.825 | Improved |
| add verifier feedback loop for initially failed forecasts (ablation) | TFRBench data generation pipeline · 10 datasets | rescue rate (% of MASE>=1 fixed): ~6.5% to ~52.4% across datasets | Improved |
Closest papers by meaning
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities (ICLR 2025 · similarity 0.88)
- Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting (NeurIPS 2026 · similarity 0.88)
- Introducing FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark (NeurIPS 2025 · similarity 0.87)
- TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop (NeurIPS 2025 · similarity 0.87)
- Context is Key: A Benchmark for Forecasting with Essential Textual Information (ICML 2025 · similarity 0.87)
- fev-bench: A Realistic Benchmark for Time Series Forecasting (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
