Research map / Time series / Foundation models for time series
Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Tiny Time Mixers (TTM), a 1–5M parameter TSMixer-based pre-trained multivariate forecasting model with adaptive patching, diverse resolution sampling, and resolution prefix tuning, achieving strong zero/few-shot forecasting on public TS datasets.
Problem
Large pre-trained time-series forecasting models are slow, computationally expensive, and often ignore cross-channel and exogenous correlations, and pre-training on heterogeneous multi-resolution TS data with small models is hard.
Approach
A tiny TSMixer-based backbone pre-trained channel-independently on ~1B samples from Monash and LibCity, made multi-resolution-capable via adaptive patching, diverse resolution sampling (downsampling high-resolution data), and a learnable resolution prefix; channel correlations and exogenous signals are added only during fine-tuning through a decoder channel-mixer and an exogenous mixer.
Key results
- TTM-Advanced (5M params) outperforms Moirai variants by 4-10% and TimesFM by 19% in zero-shot forecasting.
- TTM-Base (1M params) outperforms Chronos by 17-32% and Lag-Llama by about 40% on last-test-window zero-shot comparisons.
- TTM-CM outperforms competitive models trained from scratch on exogenous/cross-channel datasets by 15-44%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TTM-A instead of Moirai variants | TTM-A · D1 zero-shot | forecast accuracy: 4-10% | Helped |
| Use TTM-A instead of TimesFM | TTM-A · D1 zero-shot | forecast accuracy: 19% | Helped |
| Use TTM-B instead of Chronos | TTM-B · D1 zero-shot, last test window | forecast accuracy: 17-32% | Helped |
| Use TTM-B instead of Lag-Llama | TTM-B · D1 zero-shot, last test window | forecast accuracy: 40% | Helped |
| Use TTM-B instead of GPT4TS | TTM-B · D1 few-shot 5% | forecast accuracy: 15% | Helped |
| Use TTM-B instead of Time-LLM | TTM-B · D1 few-shot 5% | forecast accuracy: 10% | Helped |
| Use TTM instead of Moment for head probing | TTM · D1 full-shot head probing | forecast accuracy: 3-4% | Helped |
| Enable decoder channel mixing and the exogenous mixer | TTM · D2 full-data fine-tuning | forecast accuracy: 15-44% | Helped |
| Add diverse resolution sampling to Monash pre-training (ablation) | TTM · Zero-shot, Monash pre-training | forecast performance: 37% | Helped |
| Increase pre-training data from 250M to 1B samples (ablation) | TTM · Zero-shot | forecast performance: 6% | Helped |
| Add resolution prefix tuning (ablation) | TTM · Zero-shot, 1B pre-training samples | forecast performance: 3% | Helped |
| Add resolution prefix tuning (ablation) | TTM · Zero-shot, very short context | forecast performance: 8% | Helped |
Surprising findings
- Limited pre-training data with high resolution diversity matters more than raw data volume; DRS improved results by 37% over the smaller Monash-only setup.
Limitations
- Pre-trained for a fixed forecast length in direct mode; other lengths need adaptation (pruning or recursive).
- Backbone is channel-independent, so cross-channel modeling relies on fine-tuning the decoder.
- Some pre-training LibCity datasets were excluded from enterprise-use release variants.
Experimental setup
Datasets: Monash LibCity ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic Bike Sharing Carbon Capture Plant Application (APP)
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic Bike Sharing Carbon Capture Plant APP SER
Metrics: MSE forecast improvement percentage size improvement ratio
Baselines: Moirai TimesFM Chronos Lag-Llama Moment GPT4TS Time-LLM LLMTime PatchTST TSMixer
Builds on: TSMixer PatchTST MLP-Mixer prefix tuning
Closest papers by meaning
- xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories (NeurIPS 2025 · similarity 0.89)
- SEMPO: Lightweight Foundation Models for Time Series Forecasting (NeurIPS 2025 · similarity 0.87)
- Multi-Scale Finetuning for Encoder-based Time Series Foundation Models (NeurIPS 2025 · similarity 0.87)
- TiRex-2: Generalizing TiRex to Multivariate Data and Streaming (NeurIPS 2026 · similarity 0.87)
- TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data (AAAI 2025 · similarity 0.86)
- TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.