Research map / Time series / Foundation models for time series
TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TimesURL, a self-supervised time series representation framework combining frequency-temporal augmentation, double Universum hard negatives, and time reconstruction, evaluated on six downstream tasks.
Problem
Learning universal time series representations that transfer across forecasting, imputation, classification, anomaly detection and transfer learning, given that CV/NLP contrastive augmentations and easy negatives are ill-suited to time series.
Approach
Combines a frequency-temporal augmentation (cropping plus frequency mixing) that preserves temporal properties, anchor-specific mixup "Universum" hard negatives along temporal and instance axes, and a masked time reconstruction loss so that both segment- and instance-level information is learned jointly.
Key results
- Average accuracy 0.752 on 30 UEA datasets vs 0.714 for InfoTS (classification SOTA)
- Average accuracy 0.845 on 128 UCR datasets vs 0.838 for InfoTS
- Univariate forecasting average MSE 0.0977 and MAE 0.2292, best among compared representation and end-to-end methods
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add frequency mixing (ablation) | TimesURL · 30 UEA datasets | Avg. Accuracy: 0.709 -> 0.752 | Helped |
| add instance Universum (ablation) | TimesURL · 30 UEA datasets | Avg. Accuracy: 0.720 -> 0.752 | Helped |
| add temporal Universum (ablation) | TimesURL · 30 UEA datasets | Avg. Accuracy: 0.717 -> 0.752 | Helped |
| add double Universum (ablation) | TimesURL · 30 UEA datasets | Avg. Accuracy: 0.716 -> 0.752 | Helped |
| add time reconstruction (ablation) | TimesURL · 30 UEA datasets | Avg. Accuracy: 0.735 -> 0.752 | Helped |
| use TimesURL instead of InfoTS | time-series classification · 30 UEA datasets | Avg. ACC: 0.714 -> 0.752 | Helped |
| use TimesURL instead of InfoTS | time-series classification · 128 UCR datasets | Avg. ACC: 0.838 -> 0.845 | Helped |
| use TimesURL instead of CoST | ETT forecasting · ETT forecasting (Avg.) | MSE: 0.1021 -> 0.0977 | Helped |
Surprising findings
- Adding Universums lowers the proxy task (instance discrimination) performance but raises downstream linear classification accuracy on ERing from 0.896 to 0.985
Limitations
- Several reported results are not best (e.g., some ETT horizons where CoST or TCN are better), and the paper's claim of SOTA in most cases is only partially supported
- Transfer learning evaluated on only two source domains and nine targets within the UCR archive
- Imputation and anomaly detection evaluated on limited datasets and with a few baselines
Experimental setup
Datasets: UEA archive UCR archive ETT KPI Yahoo
Benchmarks: UEA UCR ETT KPI Yahoo
Metrics: accuracy average rank MSE MAE F1 precision recall
Baselines: InfoTS TS2Vec T-Loss TNC TS-TCC TST DTW CoST Informer LogTrans N-BEATS TCN
Builds on: TS2Vec SimCLR MAE BERT Mixup
Closest papers by meaning
- Soft Contrastive Learning for Time Series (ICLR 2024 · similarity 0.88)
- Parametric Augmentation for Time Series Contrastive Learning (ICLR 2024 · similarity 0.87)
- Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning (AAAI 2025 · similarity 0.86)
- Self-Supervised Contrastive Learning for Long-term Forecasting (ICLR 2024 · similarity 0.86)
- TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (ICML 2024 · similarity 0.86)
- REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.