Research map / Time series / Foundation models for time series
Soft Contrastive Learning for Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SoftCLT, a plug-and-play soft contrastive learning strategy for time series that assigns soft (0-1) weights to instance-wise (DTW-based) and temporal (timestamp-distance-based) negative pairs, improving TS2Vec, TS-TCC and CA-TCC across classification, semi-supervised, transfer and anomaly detection tasks.
Problem
Standard hard contrastive learning for time series treats all other instances and all other timestamps as equally negative, ignoring inherent similarities among similar series and nearby timestamps, which degrades learned representations.
Approach
Replace hard positive/negative assignments with soft assignments in InfoNCE: instance-wise weights from a sigmoid of min-max normalized DTW distance on the raw data space, and temporal weights from a sigmoid of timestamp difference, combined with hierarchical contrasting whose sharpness scales with pooling depth.
Key results
- Soft CL applied to TS2Vec improves average accuracy on UCR from 82.3% to 85.0% (+2.7) and on UEA from 70.5% to 74.2% (+3.7).
- SoftCLT applied to CA-TCC raises average accuracy over twelve FD transfer scenarios from 69.66% to 80.34% (+10.68%).
- On anomaly detection, SoftCLT on TS2Vec improves Yahoo F1 from 72.3 to 74.2 and KPI F1 from 67.6 to 70.1 in the normal setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SoftCLT to CA-TCC | CA-TCC · FD transfer learning, 12 scenarios | average accuracy: 69.66 -> 80.34 (+10.68%) | Helped |
| Add SoftCLT to TS-TCC | TS-TCC · FD transfer learning, 12 scenarios | average accuracy: 67.82 -> 75.03 (+7.21%) | Helped |
| Add SoftCLT temporal contrastive loss to TS2Vec | TS2Vec · Yahoo and KPI, normal and cold-start anomaly detection | F1: Yahoo normal: 72.3 -> 74.2; KPI normal: 67.6 -> 70.1; Yahoo cold-start: 74.0 -> 76.2; KPI cold-start: 68.9 -> 70.7 | Helped |
| Use instance-wise and temporal soft assignments (ablation) | TS2Vec · UCR datasets | average accuracy: 82.3 -> 85.0 (+2.7) | Helped |
| Use instance-wise and temporal soft assignments (ablation) | TS2Vec · UEA datasets | average accuracy: 70.5 -> 74.2 (+3.7) | Helped |
| Use instance-wise soft assignments only (ablation) | TS2Vec · UCR and UEA datasets | average accuracy: UCR: 82.3 -> 83.9 (+1.6); UEA: 70.5 -> 73.0 (+2.5) | Helped |
| Use temporal soft assignments only (ablation) | TS2Vec · UCR and UEA datasets | average accuracy: UCR: 82.3 -> 83.7 (+1.4); UEA: 70.5 -> 73.8 (+3.3) | Helped |
| Replace sigmoid temporal assignment with neighbor assignment (ablation) | Temporal CL · UCR datasets | average accuracy: 83.7 -> 76.1 | Hurt |
| Replace sigmoid temporal assignment with linear assignment (ablation) | Temporal CL · UCR datasets | average accuracy: 83.7 -> 77.2 | Hurt |
| Replace sigmoid temporal assignment with Gaussian assignment (ablation) | Temporal CL · UCR datasets | average accuracy: 83.7 -> 83.5 | Hurt |
| Set the instance-wise assignment upper bound to 0.50 (ablation) | Instance-wise CL · UCR datasets | average accuracy: 83.0 (0.25), 83.9 (0.50), 83.4 (0.75), 83.1 (1.00) | Helped |
| Remove instance-wise contrastive loss (ablation) | TS2Vec + SoftCLT · Yahoo and KPI anomaly detection, normal and cold-start | F1: Normal Yahoo: 71.2 -> 74.2; normal KPI: 66.4 -> 70.1; cold-start Yahoo: 74.6 -> 76.2; cold-start KPI: 69.0 -> 69.7 | Mixed |
Surprising findings
- Suppressing the instance-wise contrastive loss gave better anomaly detection performance on average for both TS2Vec and SoftCLT.
- Soft CL methods from computer vision (NNCLR, ASCL) that compute soft assignments in embedding space degraded TS2Vec performance, while data-space soft assignments improved it.
Limitations
- DTW distance has O(T^2) complexity, which may be costly on large-scale datasets without precomputation.
- Instance-wise CL was suppressed for anomaly detection, indicating the instance-wise component is not universally beneficial.
- Seasonality is not modeled explicitly; the method relies on the assumption that adjacent timestamps are similar.
Experimental setup
Datasets: UCR archive UEA archive SleepEEG Epilepsy FD-B Gesture EMG FD HAR Wafer FordA FordB
Benchmarks: UCR archive UEA archive HAR Epilepsy Wafer FordA FordB POC StarLightCurves ElectricDevices FD transfer Yahoo
Metrics: accuracy rank macro F1 precision recall F1
Baselines: DTW-D TNC TST TS-TCC T-Loss TS2Vec TS-SD Mixing-Up CLOCS CoST LaST TF-C
Builds on: TS2Vec InfoNCE TS-TCC CA-TCC
Closest papers by meaning
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · similarity 0.88)
- Parametric Augmentation for Time Series Contrastive Learning (ICLR 2024 · similarity 0.87)
- Self-Supervised Contrastive Learning for Long-term Forecasting (ICLR 2024 · similarity 0.87)
- Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning (AAAI 2025 · similarity 0.85)
- Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference (NeurIPS 2024 · similarity 0.85)
- Explaining Time Series via Contrastive and Locally Sparse Perturbations (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.