Research map / Time series / Long-term forecasting
ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ProbTS, a unified benchmark and toolkit that evaluates point and distributional time-series forecasting across short and long horizons, analyzing decoding schemes (AR vs NAR), distribution heads, and normalization in classical and foundation models.
Problem
Existing deep time-series forecasting work focuses on isolated settings (long-term point or short-term probabilistic), and the strengths and weaknesses of universal foundation models for point and distributional forecasting across horizons are poorly understood.
Approach
A benchmark platform that unifies point and probabilistic evaluation over short and long horizons and explicitly categorizes models by distribution estimation method, decoding scheme (AR vs NAR), and normalization, supplemented by dataset characteristics (trend, seasonality, non-Gaussianity) to explain performance differences.
Key results
- Customized long-term architectures lose competitiveness in short-term forecasting, and existing probabilistic models (e.g., TimeGrad, GRU NVP) degrade notably in long-term distributional forecasting.
- AR decoding suffers error accumulation over long horizons without normalization, but RevIN substantially improves AR models in long-term forecasting, except on Traffic.
- AR-based foundation models (TimesFM, Timer) are competitive at short horizons but their advantage over NAR MOIRAI diminishes as horizon grows; MOIRAI's mixture-head distribution falls further behind CSDI as non-Gaussianity increases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply long-term point-forecasting architectures to short-term forecasting | Customized architectures · Short-term forecasting | NMAE | Hurt |
| Apply short-term probabilistic forecasting methods to long-term distributional forecasting | Probabilistic forecasting models · Long-term forecasting | CRPS | Hurt |
| Use AR decoding instead of NAR decoding | TimeGrad · Traffic | NMAE | Helped |
| Add RevIN normalization (ablation) | AR-based models · Long-term forecasting | NMAE | Helped |
| Add RevIN normalization (ablation) | GRU NVP · ETTh1 | NMAE | Helped |
| Add RevIN normalization (ablation) | Forecasting models · Traffic long-term | NMAE | Hurt |
| Replace mean scaling with RevIN (ablation) | CSDI · Weather | CRPS | Hurt |
| Add RevIN normalization (ablation) | CSDI, TimeGrad, and GRU NVP · Short-term forecasting | NMAE | No clear effect |
| Use mean scaling (ablation) | Short-term probabilistic models · Short-term forecasting | NMAE | Helped |
| Remove instance-level normalization (ablation) | TimeGrad and GRU NVP · Wikipedia, Solar, and Electricity short-term | NMAE | Hurt |
| Extend the prediction horizon | AR-based foundation models · Long-term forecasting | NMAE | Hurt |
| Use NAR decoding rather than AR decoding | Time-series foundation models · Longer forecasting horizons | NMAE | Helped |
Surprising findings
- RevIN can negatively affect AR models on Traffic, a dataset with strong seasonality but minimal trend.
- AR decoding (TimeGrad) outperforms PatchTST on the Traffic dataset, despite AR's error-accumulation drawbacks.
Limitations
- Primarily empirical analysis without theoretical grounding.
- Focus on major decisions (AR vs NAR, distribution estimation) may overlook other factors.
- Evaluation datasets may not capture the full range of real-world forecasting challenges, and foundation models' pre-training data overlap complicates comparison.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Exchange ILI Weather Solar Wikipedia
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Exchange ILI Weather Solar Wikipedia
Metrics: NMAE CRPS MAE MSE CRPS-sum
Baselines: iTransformer PatchTST TimesNet N-HiTS LTSF-Linear TimeGrad CSDI Lag-Llama TimesFM Chronos MOIRAI Timer
Builds on: RevIN TimeGrad CSDI PyTorchTS GluonTS
Closest papers by meaning
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.88)
- TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting (ICML 2025 · similarity 0.88)
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.87)
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities (ICLR 2025 · similarity 0.87)
- Unified Training of Universal Time Series Forecasting Transformers (ICML 2024 · similarity 0.87)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.