Research map / Time series / Foundation models for time series
Parametric Augmentation for Time Series Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoTCL, a contrastive time-series representation framework that learns a parametric factorization mask and invertible mask-based augmentation per instance, trained alternately with the encoder under a relevant-information objective.
Problem
Choosing effective data augmentations for contrastive learning on time series is hard because time series semantics are not human-recognizable and augmentation effectiveness varies across datasets and instances.
Approach
Factorize each instance into an informative part and task-irrelevant noise via a learned Bernoulli-concrete mask, apply a learned non-zero (lossless) mask transform to the informative part, and add random timestamp masking for extra variance; the augmentation network is trained with a Principle of Relevant Information loss (mask L0 norm plus MMD) and a temporal consistency triplet loss, alternatin
Key results
- Univariate forecasting average MSE 0.157 and MAE 0.258 versus CoST 0.168/0.271, roughly a 6.5% MSE reduction over the leading baseline.
- UEA classification average accuracy 0.742 versus InfoTS 0.730 and TS2Vec 0.704, with average rank 2.3.
- Multivariate forecasting average MSE 0.545 and MAE 0.499, improving over CoST (0.561/0.505).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply parametric augmentation instead of CoST's universal augmentation | AutoTCL with CoST backbone · Univariate forecasting (Avg.) | MSE: 0.168 -> 0.157 | Helped |
| apply parametric augmentation instead of CoST's universal augmentation | AutoTCL with CoST backbone · Multivariate forecasting (Avg.) | MSE: 0.561 -> 0.545 | Helped |
| apply AutoTCL augmentation | TS2Vec backbone · UEA classification (Avg.) | accuracy: +1.2% | Helped |
| add the factorization head (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.173 -> 0.157; 10.19% | Helped |
| add the invertible transformation head (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.216 -> 0.157; 37.6% | Helped |
| add random view noise (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.185 -> 0.157 | Helped |
| add augmentation rather than removing it (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.260 -> 0.157 | Helped |
| replace parametric augmentation with Cutout (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.157 -> 0.266 | Hurt |
| replace parametric augmentation with Jitter (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.157 -> 0.184 | Hurt |
| train the augmentation network adversarially (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.157 -> 0.212 | Hurt |
| randomly select Cutout or Jitter instead of learning parametric augmentations (ablation) | AutoTCL · Univariate forecasting (Avg.) | MSE: 0.157 -> 0.176 | Hurt |
Surprising findings
- Cutout performs worse than no augmentation on average in the univariate ablation, suggesting fixed augmentations can hurt time series contrastive learning.
- Adversarial augmentation training improves over fixed augmentations but is outperformed by the proposed relevant-information training.
Limitations
- Lora dataset is a newly introduced dataset whose availability is not stated, so results may be hard to reproduce.
- Relies on grid search over many hyperparameters per dataset.
- Forecasting uses a linear probe on frozen encoder representations.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 Electricity Weather Lora UEA
Benchmarks: ETTh1 ETTh2 ETTm1 Electricity Weather Lora UEA
Metrics: MSE MAE accuracy average rank
Baselines: TS2Vec CoST InfoTS Informer TNC TS-TCC T-Loss TF-C
Builds on: TS2Vec CoST InfoMin Principle of Relevant Information
Closest papers by meaning
- Soft Contrastive Learning for Time Series (ICLR 2024 · similarity 0.87)
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · similarity 0.87)
- Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting (ICML 2025 · similarity 0.87)
- Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference (NeurIPS 2024 · similarity 0.86)
- Graph-Aware Contrasting for Multivariate Time-Series Classification (AAAI 2024 · similarity 0.85)
- Contrastive Difference Predictive Coding (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.