Research map / Time series / Long-term forecasting
Self-Supervised Contrastive Learning for Long-term Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoCon, an autocorrelation-based contrastive loss over windows sampled across the whole series, paired with a decomposition architecture (linear short-term branch, TCN encoder plus multi-scale moving-average long-term branch) to improve long-horizon forecasting.
Problem
Sliding-window forecasters fail to capture long-term variations that extend beyond the input window, so long-horizon forecasting performance is poor.
Approach
Sample windows far apart within a mini-batch and define positive/negative pairs by the global autocorrelation at their time lag, using a relative, weighted contrastive loss that is self-supervised and needs no augmentation; combine it with a decomposition model whose long-term branch has enough capacity to learn long-term representations.
Key results
- Extended long-term forecasting: the method ranks first in 42 of the univariate settings in the extended table, versus 4 for the next best (DLinear).
- Relative to the second-best model, reduces error by about 34% on ETTh2 and 11% on Electricity; improvement on Weather is only about 3%.
- Computational cost rises only from 31.1 to 33.2 ms/iter with AutoCon, versus 365.7 ms/iter for Nonstationary Transformer and 466.1 ms/iter for TimesNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the AutoCon contrastive loss to the forecasting objective (ablation) | AutoCon decomposition model · ETTh1, output 720 | MSE: 0.082 -> 0.078 | Helped |
| Add the short-term branch (ablation) | AutoCon decomposition model · ETTh1, output 96 | MSE: 0.071 -> 0.055 | Helped |
| Add the long-term branch (ablation) | AutoCon decomposition model · ETTh2, output 2160 | MSE: 0.592 -> 0.198 | Helped |
| Add the short-term branch (ablation) | AutoCon decomposition model · ETTm2, output 4320 | MSE: 0.288 -> 0.214 | Helped |
| Add the long-term branch (ablation) | AutoCon decomposition model · ETTm2, output 4320 | MSE: 0.326 -> 0.214 | Helped |
| Add AutoCon loss (ablation) | AutoCon decomposition model · ETTh2, output 2160 | MSE: 0.236 -> 0.198 | Helped |
| Increase the long-term branch from 1 to 2 layers (ablation) | DLinear · ETTh1 | MSE: 0.1780±0.0054 -> 0.1993±0.1964 | Hurt |
| Increase the long-term branch depth without AutoCon (ablation) | AutoCon decomposition model · ETTh1 | MSE: 0.0837±0.0185 -> 0.0910±0.0188 | Hurt |
| Add AutoCon to the 1-layer long-term branch (ablation) | AutoCon decomposition model · ETTh1 | MSE: 0.0837±0.0185 -> 0.0787±0.002 | Helped |
| Add AutoCon to the 3-layer long-term branch (ablation) | AutoCon decomposition model · ETTh2 | MSE: 0.2036±0.0265 -> 0.1771 ±0.0393 | Helped |
| Add AutoCon to the 2-layer long-term branch (ablation) | AutoCon decomposition model · Electricity | MSE: 0.2889±0.0330 -> 0.2753±0.0224 | Helped |
| Use AutoCon instead of LaST | ETTm1 long-term forecasting · ETTm1, output 4320 | MSE: 0.274 -> 0.082 | Helped |
Surprising findings
- Stacking more linear layers in DLinear's long-term branch does not improve, and often slightly degrades, long-horizon performance.
Limitations
- Autocorrelation assumes linear dependencies and cannot capture non-linear long-term patterns.
- Longer input windows increase overfitting risk because fewer training windows remain, so the sliding-window limitation is not fully solved by window scaling alone.
- Protocol modifications (extended output lengths, changed splits, corrected weather values) depart from standard benchmarks, which complicates direct comparison.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Weather Exchange ILI
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Weather Exchange ILI
Metrics: MSE MAE Shape DTW Temporal DTW
Baselines: TimesNet MICN PatchTST DLinear FiLM Nonstationary Transformer FEDformer LaST CoST TS2Vec
Builds on: SupCon DLinear TSlib TS2Vec
Closest papers by meaning
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.87)
- Soft Contrastive Learning for Time Series (ICLR 2024 · similarity 0.87)
- Loss Shaping Constraints for Long-Term Time Series Forecasting (ICML 2024 · similarity 0.86)
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · similarity 0.86)
- Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting (NeurIPS 2025 · similarity 0.86)
- Contrastive Difference Predictive Coding (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.