Research map / Time series / Foundation models for time series

TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting

NeurIPS 2026 · Time series › Foundation models for time series · approach: Time series foundation models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TempoPFN, a univariate time series foundation model using GatedDeltaProduct linear RNNs with state-weaving, pre-trained only on a fully open-source synthetic data pipeline with novel generators (SDE, sawtooth, step, spikes, audio-inspired) and augmentations.

Problem

Zero-shot time series forecasting faces challenges in efficient long-horizon prediction and reproducibility, and existing synthetic-only pre-training underperforms on challenging benchmarks like Gift-Eval.

Approach

Replace windowing and non-linear RNNs with a 10-layer GatedDeltaProduct linear RNN trained in PFN fashion on a diverse synthetic prior, adding state-weaving (final hidden state of a layer initializes the next) for non-causal, parallel prediction of all future steps, plus novel generators (regime-switching OU SDE) and a cascade of augmentations.

Key results

• On Gift-Eval, TempoPFN reaches overall CRPS 0.537 (vs TabPFN-TS 0.544) and MASE 0.797 (vs 0.771), ranking 6th in CRPS.
• On fev-bench MASE leaderboard, TempoPFN ranks 6th with 60.5% win rate and 25.1% skill score, ahead of TabPFN-TS (rank 8).
• Removing the SDE generator increases Gift-ZS overall CRPS by 26% (0.578 to 0.729); removing augmentations raises CRPS from 0.577 to 0.610.

Surprising findings

• Linear RNNs without non-linear recurrence match or exceed several real-data-trained models, challenging the claim that non-linear sLSTM is needed for state tracking.
• TempoPFN is markedly more robust to missing values than TiRex, with median CRPS rising about 4% vs 11% at 90% NaNs.

Limitations

• Univariate only; multivariate extension left to future work.
• Ablations were run at 500k iterations rather than the full 3M schedule due to pretraining cost.
• Point-forecast MASE is slightly worse than TabPFN-TS on Gift-Eval, and fev-bench ranking is 6th rather than top.

Experimental setup

Datasets: synthetic generators (ForecastPFN, KernelSynth, Gaussian Process, CauKer, Sawtooth, Step Function, Anomaly, Spikes, Sine

Benchmarks: Gift-Eval Chronos-ZS fev-bench

Metrics: CRPS MASE average rank win rate skill score SQL

Baselines: TabPFN-TS TiRex Chronos Bolt B TimesFM 2.0 YingLong 50m Mamba4Cast Chronos-2 MOIRAI 2.0 Sundial-Base Chronos-Bolt

Builds on: Prior-data Fitted Networks DeltaProduct TiRex ForecastPFN CauKer TS-Mixup

Closest papers by meaning

• TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data (AAAI 2025 · similarity 0.90)
• TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.88)
• Zero-shot forecasting of chaotic systems (ICLR 2025 · similarity 0.88)
• TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning (NeurIPS 2025 · similarity 0.88)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.87)
• PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting (NeurIPS 2024 · similarity 0.87)