Research map / Time series / Foundation models for time series
Abstracted Shapes as Tokens - A Generalizable and Interpretable Model for Time-series Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains VQShape, a patch-based transformer VQ-VAE that represents time series as vector-quantized abstracted shapes with offset, scale, position and length attributes, yielding interpretable tokens and code histograms for classification and zero-shot transfer.
Problem
Existing time-series foundation models are black boxes that give no human-understandable representations, while shapelet-based interpretable models are dataset-specific and fixed-length.
Approach
Decompose each subsequence into a quantized abstracted shape code plus offset, scale, start time and relative length, and pre-train a VQ-VAE with a subsequence-reconstruction loss and a diversity regularizer so the codebook of shapes is dataset-agnostic and interpretable.
Key results
- VQShape (token representation with linear classifier) reaches mean accuracy 0.723 across 29 UEA datasets, vs 0.719 for T-Rep and 0.712 for TS2Vec, with no statistically significant differences among top methods.
- Under the same frozen linear-probe protocol, VQShape achieves mean accuracy 0.723 vs 0.697 for MOMENT and 0.581 for UniTS when pre-trained on the 29 datasets.
- VQShape pre-trained on only 9 UEA datasets still reaches mean accuracy 0.723 across all 29 datasets, showing cross-domain generalization to unseen datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VQShape representations instead of the strongest overall baselines | UEA multivariate time-series classification · UEA 29 datasets | mean accuracy: 0.719 (T-Rep) -> 0.723 (VQShape) | No clear effect |
| Use VQShape instead of MOMENT or UniTS with 29-dataset pre-training | frozen pre-trained representations · UEA 29 datasets | mean accuracy: 0.697 (MOMENT), 0.581 (UniTS) -> 0.723 | Helped |
| Increase pre-training data from 9 to 29 UEA datasets | VQShape · UEA 29 datasets | mean accuracy: 0.723 -> 0.723 | No clear effect |
| Increase code dimension from 8 to 32 (ablation) | VQShape token classifier · UEA 29 datasets | mean accuracy: 0.723 -> 0.721 | Hurt |
| Increase code dimension from 8 to 32 (ablation) | VQShape histogram classifier · UEA 29 datasets | mean accuracy: 0.709 -> 0.717 | Helped |
| Add shape-level subsequence reconstruction loss (ablation) | VQShape token classifier · UEA 29 datasets | mean accuracy: 0.708 -> 0.723 | Helped |
| Add shape-level subsequence reconstruction loss (ablation) | VQShape histogram classifier · UEA 29 datasets | mean accuracy: 0.707 -> 0.709 | Helped |
| Use codebook size 64 instead of larger codebooks (ablation) | VQShape histogram classifier · UEA datasets | mean accuracy | Helped |
| Increase codebook size (ablation) | VQShape token classifier · UEA datasets | mean accuracy | Helped |
| Use token representations instead of histogram representations (ablation) | VQShape classifiers · UEA datasets | mean accuracy | Helped |
Surprising findings
- Simply adding more pre-training data can degrade performance on some datasets (e.g., BasicMotions), since high-frequency sinusoidal signals contain no meaningful shape-level information.
- Histogram classifiers peak at codebook size 64 and decline with larger codebooks, while token classifiers improve steadily with codebook size.
Limitations
- Evaluation limited to classification; forecasting, imputation and anomaly detection not explored.
- Pre-training data is limited (about 1.4M univariate series), so the authors do not call it a foundation model.
- Histogram representation performance depends on codebook size chosen post hoc; no mechanism to set it automatically.
Experimental setup
Datasets: UEA Multivariate Time Series Classification Archive (29 datasets)
Benchmarks: UEA Multivariate Time Series Classification Archive (29 datasets)
Metrics: classification accuracy mean accuracy median accuracy mean rank Wilcoxon p-value
Baselines: DTW STRF DLinear Autoformer FEDformer PatchTST TimesNet TS-TCC TST T-Rep TS2Vec MOMENT
Builds on: VQ-VAE MOMENT shapelet transform concept bottleneck models BOSS
Closest papers by meaning
- Diffusion Language-Shapelets for Semi-supervised Time-Series Classification (AAAI 2024 · similarity 0.89)
- ShapeX: Shapelet-Driven Post Hoc Explanations for Time Series Classification Models (NeurIPS 2025 · similarity 0.89)
- Learning Soft Sparse Shapes for Efficient Time-Series Classification (ICML 2025 · similarity 0.88)
- Time Series Representations with Hard-Coded Invariances (ICML 2025 · similarity 0.86)
- Shape analysis for time series (NeurIPS 2024 · similarity 0.85)
- T-Rep: Representation Learning for Time Series using Time-Embeddings (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.