Research map / Time series / Long-term forecasting
Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AMRC, a plug-in training loss (adaptive prefix-masking loss plus an embedding-similarity penalty) that suppresses redundant input features and improves multivariate time series forecasting across five backbone models.
Problem
Existing deep time series forecasters are assumed to benefit from longer inputs, yet truncating historical prefixes often improves MSE, suggesting they learn redundant features that hurt generalization.
Approach
Use information-bottleneck motivation to add an adaptive masking loss that pulls the encoder representation of the full input toward the representation of the best randomly prefix-masked input (weighted by the relative loss gain), plus an embedding-similarity penalty that aligns pairwise embedding distances with pairwise output distances within a batch.
Key results
- AMRC lowers SOFTS MSE on ETTh1 from 0.408 to 0.389 and iTransformer MSE on Electricity from 0.176 to 0.163 (averages over horizons).
- On ETTm2, PatchTST MSE drops from 0.211 to 0.196 (about 7%).
- Ablation shows AML-only and ESP-only each give gains, with the combined AMRC best on most settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add AMRC to SOFTS | SOFTS · ETTh1 | MSE: 0.408 -> 0.389 | Helped |
| Add AMRC to iTransformer | iTransformer · Electricity | MSE: 0.176 -> 0.163 | Helped |
| Add AMRC to PatchTST | PatchTST · ETTm2 | MSE: 0.211 -> 0.196 | Helped |
| Add AMRC to TimeMixer | TimeMixer · Weather | MSE: 0.197 -> 0.186 | Helped |
| Add AMRC to SOFTS | SOFTS · ETTh2 | MAE: 0.359 -> 0.362 | Hurt |
| Add AML without ESP (ablation) | SOFTS · ETTh1 | MSE: 0.408 -> 0.401 | Helped |
| Add ESP without AML (ablation) | SOFTS · ETTh1 | MSE: 0.408 -> 0.393 | Helped |
| Add AML without ESP (ablation) | iTransformer · Weather | MSE: 0.209 -> 0.205 | Helped |
| Add ESP without AML (ablation) | iTransformer · Weather | MSE: 0.209 -> 0.210 | No clear effect |
| Add AML without ESP (ablation) | TimeMixer · ETTh2 | MSE: 0.318 -> 0.319 | No clear effect |
| Add ESP without AML (ablation) | TimeMixer · ETTh2 | MSE: 0.318 -> 0.317 | Helped |
| Combine AML and ESP (ablation) | SOFTS · ETTh1 | MSE: 0.401 -> 0.389 | Helped |
Surprising findings
- Truncating (masking) early input segments often lowers MSE for existing models, contrary to the long-sequence information gain hypothesis.
Limitations
- AML is inactive when the prefix contains no redundant information (beta stays zero), and the masking process is a black box.
- Training cost grows roughly m-fold due to mask sampling.
- ESP gains are smaller on high-dimensional datasets such as Weather and Solar-Energy.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Solar-Energy Electricity Weather
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Solar-Energy Electricity Weather
Metrics: MSE MAE
Baselines: SOFTS iTransformer PatchTST TSMixer TimeMixer
Builds on: information bottleneck Temporal Ensembling Mean Teacher
Closest papers by meaning
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.87)
- Loss Shaping Constraints for Long-Term Time Series Forecasting (ICML 2024 · similarity 0.86)
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.86)
- Language Pre-training Guided Masking Representation Learning for Time Series Classification (AAAI 2025 · similarity 0.86)
- Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.86)
- Learning to Embed Time Series Patches Independently (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.