Research map / Time series / Foundation models for time series
Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S3, a plug-and-play layer that segments a time series, learns a shuffle order via differentiable permutation, and stitches it back with a learned weighted sum with the original input, improving classification, forecasting, and anomaly detection baselines.
Problem
Standard time-series representation learning keeps the original temporal order fixed, even though distant segments may have strong dependencies that convolutional and Transformer models capture poorly.
Approach
Insert a learnable segment-shuffle layer at the input: split the series into non-overlapping segments, reorder them by learned priority scores through a differentiable permutation trick, and combine the reordered sequence with the original via a learned Conv1D or weighted sum; layers can be stacked with increasing segment counts.
Key results
- Adding S3 to TS2Vec, DSN, InfoTS and SoftCLT improves classification accuracy on UCR, UEA, EEG, EEG2 and HAR, with per-dataset gains up to 39.59% (InfoTS on EEG).
- On univariate forecasting, Informer+S3 reduces MSE by 16.69% on average, with a 68.71% reduction at ETTh1 horizon 720.
- On anomaly detection, SoftCLT+S3 reaches F1 0.7498 on Yahoo and 0.6892 on KPI, slightly above SoftCLT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add S3 to TS2Vec | TS2Vec · UCR | accuracy: 0.819 -> 0.851 | Helped |
| Add S3 to DSN | DSN · UEA | accuracy: 0.687 -> 0.736 | Helped |
| Add S3 to SoftCLT | SoftCLT · Yahoo | F1: 0.7399 -> 0.7498 | Helped |
| Add S3 to SoftCLT | SoftCLT · KPI | F1: 0.6890 -> 0.6892 | Helped |
| Add S3 to Informer | Informer · Univariate forecasting average | MSE: 0.2124 -> 0.1769; 16.69% | Helped |
| Add S3 to PatchTST | PatchTST · Multivariate forecasting average | MSE: 0.2692 -> 0.2648; 1.62% | Helped |
| Add S3 to MOMENT | MOMENT · PTB-XL | accuracy: 0.7176 -> 0.7552; 5.24% | Helped |
| Add Segment to S3 (ablation) | TS2Vec+S3 · EEG | accuracy: 0.552 -> 0.672 | Helped |
| Add learned shuffling parameters P to S3 (ablation) | TS2Vec+S3 · EEG | accuracy: 0.562 -> 0.672 | Helped |
| Add Stitch to S3 (ablation) | TS2Vec+S3 · EEG | accuracy: 0.584 -> 0.672 | Helped |
| Add S3 to TS2Vec | TS2Vec · ETTm1, horizon 24 | MSE: 0.0150 -> 0.0152; -1.74% | Hurt |
| Add S3 to LaST | LaST · ETTh1, horizon 24, multivariate | MSE: 0.3230 -> 0.3249; -0.55% | Hurt |
Surprising findings
- Gains are largest for weaker baselines such as Informer, while PatchTST improves by only about 1.6% MSE on average.
- On some datasets such as EEG the accuracy gain is large, but sub-optimal common hyperparameters still give consistent improvements.
Limitations
- Evaluation of other tasks such as imputation is left for future work.
- Applying S3 to other time-series forms such as video is not explored.
- Optimal number of segments n has no general rule and must be tuned per dataset.
Experimental setup
Datasets: UCR archive UEA archive EEG EEG2 HAR PTB-XL ETTh1 ETTh2 ETTm1 Electricity Weather Yahoo
Benchmarks: UCR archive UEA archive EEG EEG2 HAR ETTh1 ETTh2 ETTm1 Electricity Weather Yahoo KPI
Metrics: accuracy MSE MAE F1 training loss standard deviation
Baselines: TS2Vec DSN InfoTS SoftCLT LaST Informer PatchTST CoST MOMENT
Builds on: TS2Vec SoftCLT PatchTST Informer LaST
Closest papers by meaning
- T-Rep: Representation Learning for Time Series using Time-Embeddings (ICLR 2024 · similarity 0.88)
- Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning (ICML 2024 · similarity 0.87)
- Learning to Embed Time Series Patches Independently (ICLR 2024 · similarity 0.86)
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.86)
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.86)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.