Research map / Time series / Foundation models for time series
Time series representation learning
Learn time series embeddings with contrastive and non-contrastive objectives and augmentation.
115 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Foundation models for time series, keyword cluster “contrastive · classification · supervised”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 5 of 6 tracked benchmarks.
43 papers in 2023–24 and 49 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 40.2% to 19.3%.
Open questions the data points to
- ETTh1 (MSE): best went from 0.425 (up to 2024) to 0.4 in 2025 — Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations)
- ETTh2 (MSE): best went from 0.371 (up to 2024) to 0.347 in 2025 — Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations)
- ETTm1 (MSE): best went from 0.371 (up to 2024) to 0.336 in 2025 — Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations)
- ETTm2 (MSE): best went from 0.279 (up to 2024) to 0.254 in 2025 — Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations)
- Traffic (MSE): best went from 0.42 (up to 2024) to 0.378 in 2025 — Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations)
- Papers report TS2Vec on Multivariate forecasting, Avg. (MSE) anywhere from 0.697 to 0.852: comparisons against it depend on whose number you take.
- Papers report TS2Vec on Univariate forecasting, Avg. (MSE) anywhere from 0.175 to 0.207: comparisons against it depend on whose number you take.
What looks settled
- Sleep-EDF (accuracy): best up to 2024 87.45, best in 2025–26 84.95 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Deep Canonical Time Warping (CVPR 2016 · 44 citations)
- 2017 · Soft-DTW: a Differentiable Loss Function for Time-Series (ICML 2017 · 137 citations)
- 2018 · Non-Linear Temporal Subspace Representations for Activity Recognition (CVPR 2018 · 50 citations)
- 2019 · Unsupervised Scalable Representation Learning for Multivariate Time Series (NeurIPS 2019 · 232 citations)
- 2020 · N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (ICLR 2020 · 164 citations)
- 2021 · Time Series Domain Adaptation via Sparse Associative Structure Alignment (AAAI 2021 · 85 citations)
- 2022 · TS2Vec: Towards Universal Representation of Time Series (AAAI 2022 · 783 citations)
- 2023 · Time Series Contrastive Learning with Information-Aware Augmentations (AAAI 2023 · 100 citations)
- 2024 · UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition (CVPR 2024 · 328 citations)
- 2025 · Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification (AAAI 2025 · 6 citations)
- 2026 · Mask the Redundancy: Evolving Masking Representation Learning for Multivariate Time-Series Clustering (AAAI 2026 · 5 citations)
Earliest papers here
- Deep Canonical Time Warping (CVPR 2016 · 44 citations)
- Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA (NeurIPS 2016)
- Soft-DTW: a Differentiable Loss Function for Time-Series (ICML 2017 · 137 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Sleep-EDF · accuracy (higher is better)
5 papers, 2022–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 87.45 | Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion (ICML 2022 · 30 citations) |
ETTh1 · MSE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.425 | TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · 1 citation) |
| 2025 | 0.4 | Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations) |
ETTh2 · MSE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.371 | TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · 1 citation) |
| 2025 | 0.347 | Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations) |
ETTm1 · MSE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.371 | TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · 1 citation) |
| 2025 | 0.336 | Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations) |
ETTm2 · MSE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.279 | TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · 1 citation) |
| 2025 | 0.254 | Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations) |
Traffic · MSE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.42 | TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · 1 citation) |
| 2025 | 0.378 | Enhancing Masked Time-Series Modeling via Dropping Patches (AAAI 2025 · 4 citations) |
What has been tried
80 papers here report 882 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 5%, hurt 19%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add an SE Block with 1/4 channel reduction | ImageNet-1K and ADE20K · ADE20K mIoU | 45.1 -> 46.5 | Helped | UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition (CVPR 2024 · 328 citations) |
| add frequency mixing | 30 UEA datasets · Avg. Accuracy | 0.709 -> 0.752 | Helped | TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · 107 citations) |
| fine-tune the pretrained model rather than train with supervised learning | NYSE · IRR | 0.077 -> 0.789 | Helped | CI-STHPAN: Pre-trained Attention Network for Stock Selection with Channel-Independent Spatio-Temporal Hypergraph (AAAI 2024 · 41 citations) |
| Add node augmentation | HAR · accuracy | 92.97±0.23 -> 94.27±0.12 | Helped | Graph-Aware Contrasting for Multivariate Time-Series Classification (AAAI 2024 · 25 citations) |
| Remove instance-level cross-domain contrast | UCR 40 datasets · Clustering performance (NMI and RI) | Hurt | Cross-Domain Contrastive Learning for Time Series Clustering (AAAI 2024 · 13 citations) | |
| add random preservation map (uniform U(0,1)) to Jittering | HAR · accuracy | -0.4 | Hurt | SimPSI: A Simple Strategy to Preserve Spectral Information in Time Series Data Augmentation (AAAI 2024 · 12 citations) |
| Add the dual-constraint frequency loss to temporal reconstruction | HAR linear evaluation · accuracy | 89.47 -> 91.68 | Helped | Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification (AAAI 2025 · 6 citations) |
| Add complementary contrastive learning | Multivariate time series forecasting · MSE | 0.304 -> 0.301 | Helped | Learning to Embed Time Series Patches Independently (ICLR 2024 · 6 citations) |
| Add the target embedding inference branch | EMG · precision | 57.69 -> 90.20 | Helped | Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning (AAAI 2025 · 5 citations) |
| Use instance-wise and temporal soft assignments | UCR datasets · average accuracy | 82.3 -> 85.0 (+2.7) | Helped | Soft Contrastive Learning for Time Series (ICLR 2024 · 4 citations) |
| add multi-period frequency learning | UCIHAR, HHAR-P, WISDM · accuracy | 66.88 -> 73.47; 51.08 -> 53.16; 56.47 -> 59.10 | Helped | Boosting Transferability and Discriminability for Time Series Domain Adaptation (NeurIPS 2024 · 4 citations) |
| add cross-version contrastive loss instead of using SupCon | All versions · F1 | 2.45% | Helped | CLEP: A Novel Contrastive Learning Method for Evolutionary Reentrancy Vulnerability Detection (AAAI 2025 · 2 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| TS2Vec | Multivariate forecasting, Avg. | MSE | 0.697 – 0.852 | 2 papers, e.g. ICLR 2024 and AAAI 2023 |
| TS2Vec | Univariate forecasting, Avg. | MSE | 0.175 – 0.207 | 2 papers, e.g. AAAI 2023 and ICLR 2024 |
What reviewers will expect
Datasets and baselines named in the 30 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HHAR · 5 of 30 papers
- HAR · 3 of 30 papers
- Weather · 3 of 30 papers
- Traffic · 3 of 30 papers
- WISDM · 3 of 30 papers
- UCIHAR · 3 of 30 papers
- SelfRegulationSCP1 · 2 of 30 papers
- ETTh1 · 2 of 30 papers
Baselines
- TS2Vec · 6 of 30 papers
- iTransformer · 5 of 30 papers
- PatchTST · 4 of 30 papers
- SimMTM · 4 of 30 papers
- TS-TCC · 3 of 30 papers
- DLinear · 3 of 30 papers
- TF-C · 3 of 30 papers
- Transformer · 3 of 30 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 92 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 7 papers
- Shandong University · 5 papers
- Harbin Institute of Technology · 5 papers
- Nanyang Technological University · 4 papers
- Korea Advanced Institute of Science and Technology · 4 papers
- ETHZ - ETH Zurich · 4 papers
- University of Illinois, Urbana Champaign · 4 papers
- A*STAR · 3 papers
- Florida International University · 3 papers
- Amazon · 3 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Time series classification (61 papers)
- Satellite image time series (28 papers)
Elsewhere in the topic
- LLM-based time series forecasting (90 papers)
- Time series foundation models (71 papers)
- Time-series language reasoning (44 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).