Research map / Time series / Foundation models for time series
Towards Enhancing Time Series Contrastive Learning: A Dynamic Bad Pair Mining Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies noisy and faulty positive pairs in time series contrastive learning and proposes DBPM, a memory-based, loss-history-driven plug-in that down-weights them, improving SimCLR, MoCo, SimSiam, TSTCC, TF-C and BTSF on four datasets.
Problem
Positive pairs created by augmentation in time series contrastive learning can be noisy (dominated by noise) or faulty (augmentation destroys key temporal patterns), which misleads the encoder during representation learning.
Approach
A memory module tracks each positive pair's loss history across epochs; pairs whose historical mean loss falls outside mean ± sigma thresholds are flagged as noisy (low loss) or faulty (high loss) and down-weighted by a Gaussian PDF transformation of their current loss, with no learnable parameters.
Key results
- On PTB-XL diagnostic classification, BTSF improves from 69.48 to 73.32 AUROC with DBPM, and TF-C rhythm rises from 80.46 to 82.10.
- On HAR, DBPM with SimCLR reaches 90.42 accuracy vs 90.20 baseline, and with BTSF reaches 85.70 vs 84.24.
- DBPM adds only about 0.3-1.5 s/epoch run-time and 0.57-2.60 MB extra memory across datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DBPM to contrastive-learning baselines | SimCLR, MoCo, SimSiam, TSTCC, TF-C, and BTSF · PTB-XL Diagnostic, Form, and Rhythm Classification | AUROC: 69.48 ± 2.52 -> 73.32 ± 3.08; 80.46 ± 2.69 -> 82.10 ± 1.88; 80.18 ± 5.22 -> 80.89 ± 3.85 | Mixed |
| Add DBPM to contrastive-learning baselines | SimCLR, MoCo, SimSiam, TSTCC, TF-C, and BTSF · HAR, Sleep-EDF, and Epilepsy | Accuracy and AUPRC | Mixed |
| Add DBPM to SimCLR | SimCLR · PTB-XL Rhythm Classification | AUROC: 80.18 ± 5.22 -> 80.89 ± 3.85 | Helped |
| Add DBPM to TF-C | TF-C · PTB-XL Rhythm Classification | AUROC: 80.46 ± 2.69 -> 82.10 ± 1.88 | Helped |
| Add DBPM to BTSF | BTSF · PTB-XL Diagnostic Classification | AUROC: 69.48 ± 2.52 -> 73.32 ± 3.08 | Helped |
| Remove memory module M (ablation) | SimCLR + DBPM · PTB-XL Rhythm Classification | AUROC: 80.89 ± 3.85 -> 79.94 ± 3.75 | Hurt |
| Remove memory module M (ablation) | SimCLR + DBPM · PTB-XL Form Classification | AUROC: 78.07 ± 1.01 -> 77.62 ± 1.83 | Hurt |
| Remove memory module M (ablation) | SimCLR + DBPM · PTB-XL Diagnostic Classification | AUROC: 81.56 ± 1.84 -> 81.54 ± 1.82 | No clear effect |
| Remove transformation module T and set identified-pair weights to zero (ablation) | SimCLR + DBPM · PTB-XL Rhythm Classification | AUROC: 80.89 ± 3.85 -> 80.54 ± 3.69 | Hurt |
| Remove transformation module T and set identified-pair weights to zero (ablation) | SimCLR + DBPM · PTB-XL Form Classification | AUROC: 78.07 ± 1.01 -> 77.72 ± 1.08 | Hurt |
| Remove transformation module T and set identified-pair weights to zero (ablation) | SimCLR + DBPM · PTB-XL Diagnostic Classification | AUROC: 81.56 ± 1.84 -> 81.21 ± 2.05 | Hurt |
| Use constant weight w=0.50 for identified bad pairs (ablation) | DBPM · Sleep-EDF | Accuracy: 82.82±0.74 -> 82.80±0.76 | Hurt |
Surprising findings
- RINCE, a robust InfoNCE loss designed for faulty views, sometimes degrades performance versus the base method (e.g., MoCo on PTB-XL diagnostic 76.41 to 72.85).
- Noisy positive pairs show small contrastive loss during training, so low loss does not necessarily indicate useful learning.
Limitations
- Thresholds beta_np and beta_fp are hand-set heuristics in [1,3] rather than automatically selected.
- Only augmentation-based time series contrastive learning is studied; non-augmentation methods such as TS2Vec are excluded.
- Gains on some settings are small or within run-to-run variance (e.g., BTSF rhythm and TSTCC variants).
Experimental setup
Datasets: PTB-XL HAR Sleep-EDF Epilepsy simulated sine/square/sawtooth dataset
Benchmarks: PTB-XL HAR Sleep-EDF Epilepsy
Metrics: AUROC accuracy AUPRC
Baselines: SimCLR MoCo SimSiam TSTCC TF-C BTSF RINCE
Builds on: SimCLR MoCo SimSiam TS-TCC TF-C RINCE
Closest papers by meaning
- Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning (AAAI 2025 · similarity 0.85)
- REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning (ICLR 2024 · similarity 0.85)
- Soft Contrastive Learning for Time Series (ICLR 2024 · similarity 0.84)
- Causality-Aware Contrastive Learning for Robust Multivariate Time-Series Anomaly Detection (ICML 2025 · similarity 0.84)
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · similarity 0.84)
- Self-Supervised Contrastive Learning for Long-term Forecasting (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.