Research map / Time series / Foundation models for time series
TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TimePerceiver, an encoder-decoder for multivariate time-series forecasting that generalizes the task to arbitrary (extrapolation, interpolation, imputation) input/target index sets, using latent-bottleneck encoding and query-based decoding.
Problem
Time-series forecasting models focus on encoder design and use simple linear decoders and separate pretraining-finetuning, which are misaligned; standard forecasting assumes fixed contiguous lookback and horizon windows.
Approach
Generalize forecasting to arbitrary, possibly non-contiguous input and target time indices trained with MSE on randomly sampled targets; encode inputs into a small set of latent bottleneck tokens and decode via learnable target-timestamp queries cross-attending to encoded inputs.
Key results
- Achieves best MSE/MAE in 55 of 80 settings (17 second-best) averaged over input lengths 96/384/768, with average rank 1.375 (MSE) and 1.550 (MAE).
- Versus CARD, reports 5.6% lower average MSE and 4.4% lower MAE; versus iTransformer, 8.5% lower MSE and 7.3% lower MAE.
- Generalized formulation improves over standard formulation by 5.0% MSE and 3.4% MAE on average in ablations (L=384).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with the generalized formulation instead of the standard future-only formulation (ablation) | TimePerceiver · ETTh1, ETTm1, Solar, and ECL | MSE: 0.420 -> 0.404; 0.355 -> 0.338; 0.194 -> 0.182; 0.169 -> 0.157 | Helped |
| Train with the generalized formulation instead of the standard future-only formulation (ablation) | TimePerceiver · ETTh1, ETTm1, Solar, and ECL | MAE: 0.437 -> 0.420; 0.382 -> 0.370; 0.243 -> 0.233; 0.265 -> 0.249 | Helped |
| Use a latent bottleneck instead of full or decoupled self-attention (ablation) | TimePerceiver · ETTh1, ETTm1, Solar, ECL | MSE: ETTh1: 0.425/0.423 -> 0.404; ETTm1: 0.353/0.356 -> 0.338; Solar: 0.192/0.189 -> 0.182; ECL: 0.161/0.158 -> 0.157 | Helped |
| Share positional embeddings between the encoder and decoder (ablation) | TimePerceiver · ETTh1, ETTm1, ECL | MSE: 0.423 -> 0.404; 0.342 -> 0.338; 0.163 -> 0.157 | Helped |
| Share positional embeddings between the encoder and decoder (ablation) | TimePerceiver · Solar | MAE: 0.232 -> 0.233 | Hurt |
| Use a query-based decoder that attends to updated input representations rather than decoding directly from latents (ablation) | TimePerceiver · ETTh1 | MSE: 0.433 -> 0.404 | Helped |
| Use a query-based decoder that attends to updated input representations rather than decoding directly from latents (ablation) | TimePerceiver · ETTh1 | MAE: 0.441 -> 0.420 | Helped |
| Replace PatchTST's standard formulation and linear decoder with the generalized formulation and query-based decoder (ablation) | PatchTST encoder · ETTh1 | MSE: 0.423 -> 0.415 | Helped |
| Replace PatchTST's standard formulation and linear decoder with the generalized formulation and query-based decoder (ablation) | PatchTST encoder · ETTh1 | MAE: 0.436 -> 0.431 | Helped |
| Use fully contiguous, fully disjoint, or mixed target-patch sampling (ablation) | TimePerceiver generalized formulation · Forecasting benchmarks | MSE | Mixed |
Surprising findings
- Using latent tokens directly for decoding (DirectLT) degrades ETTh1 MSE from 0.404 to 0.433, indicating re-expansion to input resolution matters.
Limitations
- Gains on Traffic and some datasets are smaller; the best configuration of target sampling varies by dataset and requires selection.
- Results are reported as averages over input lengths and seeds on standard benchmarks only; no evaluation on other domains stated.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Solar Electricity Traffic
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Solar Electricity Traffic
Metrics: MSE MAE average rank
Baselines: DeformableTST CARD CATS PatchTST iTransformer S-Mamba DLinear TimesNet TiDE Crossformer
Builds on: Perceiver PatchTST RevIN Vision Transformer
Closest papers by meaning
- Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.90)
- A decoder-only foundation model for time-series forecasting (ICML 2024 · similarity 0.89)
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.89)
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.89)
- SteerCast: Retrieval-Based Latent Steering for Decoder-Only Time Series Forecasting (NeurIPS 2026 · similarity 0.89)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.