Research map / Time series / Long-term forecasting
Selective Learning for Deep Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes selective learning, which computes the MSE loss only on a dynamically chosen subset of timesteps (via an entropy-based uncertainty mask and a residual-lower-bound anomaly mask) to reduce overfitting in deep time series forecasting.
Problem
Deep time series forecasting models overfit because uniform MSE optimization over all timesteps forces them to fit uncertain (noisy) and anomalous timesteps that do not generalize.
Approach
Train on a screened subset of timesteps: mask high-entropy residual timesteps (uncertainty mask, estimated from residuals across sliding-window predictions over training) and timesteps whose residual is close to an estimated residual lower bound (anomaly mask, from a lightweight DLinear estimator), with masks recomputed dynamically during training.
Key results
- Selective learning reduces Informer's MSE by 37.4% on average across datasets (66.8% on ETTm2).
- On iTransformer, selective learning gives 6.5% average MSE reduction; on TimesNet 8.4%.
- In zero-shot ETTh2→ETTh1 and ETTm2→ETTm1 transfer, selective learning reduces MSE by 22.6% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply selective learning | Informer · ETTh1 and ETTm2 | MSE: -58.3%; -66.8% | Helped |
| apply selective learning | Crossformer · ETTh1 and ETTm2 | MSE: -5.33%; -37.1% | Helped |
| apply selective learning | PatchTST · ETTh1 and ETTm2 | MSE: -4.10%; -6.83% | Helped |
| apply selective learning | TimesNet · ETTh1 and ETTm2 | MSE: -14.0%; -10.7% | Helped |
| apply selective learning | iTransformer · ETTh1 and ETTm2 | MSE: -9.29%; -6.40% | Helped |
| apply selective learning | TimeMixer · ETTh1 and ETTm2 | MSE: -7.11%; -5.01% | Helped |
| add uncertainty masking (ablation) | iTransformer · ETTh1 | MSE: 0.436 -> 0.415 | Helped |
| add anomaly masking (ablation) | iTransformer · ETTh1 | MSE: 0.431 -> 0.415 | Helped |
| replace random masking with the dual-mask mechanism (ablation) | iTransformer · ETTh1 | MSE: 0.457 -> 0.415 | Helped |
| replace static masking with dynamic masking (ablation) | iTransformer · ETTh1 | MSE: 0.426 -> 0.415 | Helped |
| add uncertainty masking (ablation) | iTransformer · Synthetic dataset | MSE: 0.0295 -> 0.0475; 0.1569; 0.1640 | Hurt |
| add anomaly masking (ablation) | iTransformer · Synthetic dataset | MSE: 0.0295 -> 0.0299; 0.0294; 0.0293 | Mixed |
Surprising findings
- Uncertainty masking degrades performance on a clean synthetic dataset by misidentifying noise, while anomaly masking maintains or slightly improves it.
- Random masking at the same ratio yields performance comparable to or worse than unmasked training.
Limitations
- May filter out rare extreme events, compromising extreme-event forecasting.
- Scope limited to in-domain forecasting; not directly compatible with time series foundation model pretraining.
- Uncertainty masking can hurt on clean data; requires per-dataset masking ratio tuning.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Exchange Weather ILI
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Exchange Weather ILI
Metrics: MSE MAE
Baselines: Informer Crossformer PatchTST TimesNet iTransformer TimeMixer TILDE-Q FreDF PS loss MTGNN
Builds on: MSE training curriculum learning RevIN
Closest papers by meaning
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.87)
- Multi-Scale Finetuning for Encoder-based Time Series Foundation Models (NeurIPS 2025 · similarity 0.86)
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.86)
- FilterNet: Harnessing Frequency Filters for Time Series Forecasting (NeurIPS 2024 · similarity 0.86)
- SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting (ICML 2024 · similarity 0.86)
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.