Research map / Time series / Foundation models for time series

Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting

NeurIPS 2025 · Time series › Foundation models for time series · approach: Time series representation learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SCAM, a self-supervised re-labeling method that uses intermediate outputs of a reconstruction network as pseudo labels, masking overfitted label components, plus spectral norm regularization, to improve time series forecasting across backbones.

Problem

Deep time series forecasting models rely heavily on noisy, high-quality raw labels and overfit them, leaving available data under-exploited.

Approach

Co-train a simple convolution-FFN reconstruction network with the predictor, using reconstructions as pseudo labels; an adaptive mask derived from a decomposed co-objective loss replaces overfitted raw-label components, while spectral norm regularization on the first and last linear layers reduces sharpness.

Key results

• SCAM+SNR improves MSE for all four backbones on all seven main datasets; e.g., iTransformer ETTh1 MSE 0.454 to 0.431 and Traffic 0.428 to 0.411.
• On ETTh2, PatchTST MSE drops from 0.387 to 0.370 with SCAM+SNR.
• Ablation on cycleNet shows SCAM is the primary driver of gains, with SNR adding further improvement when combined.

Measured effects reported in the paper

Surprising findings

• Using SNR alone on iTransformer slightly worsens ETTh1 MSE (0.438 to 0.443), indicating it is not a standalone improvement in all cases.

Limitations

• Not verified on LLM-based or LLM-scale forecasting models due to cost.
• Binary mask may collapse too quickly on noisy real-world data.
• Reconstruction network architecture is a simple prototype not claimed to be optimal.

Experimental setup

Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Weather PeMS

Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Weather PeMS

Metrics: MSE MAE sharpness (lambda_max)

Baselines: MLP cycleNet PatchTST iTransformer

Builds on: RevIN cycleNet PatchTST iTransformer sharpness-aware minimization

Closest papers by meaning

• TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.87)
• Self-Supervised Contrastive Learning for Long-term Forecasting (ICLR 2024 · similarity 0.86)
• Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.85)
• Time-o1: Time-Series Forecasting Needs Transformed Label Alignment (NeurIPS 2025 · similarity 0.85)
• TSLANet: Rethinking Transformers for Time Series Representation Learning (ICML 2024 · similarity 0.85)
• Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift (NeurIPS 2025 · similarity 0.85)