Research map / Time series / Foundation models for time series
T-Rep: Representation Learning for Time Series using Time-Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T-Rep, a self-supervised convolutional encoder that learns timestep-level representations by adding a learned time-embedding module and two time-embedding-based pretext tasks (divergence prediction and time-conditioned forecasting) on top of TS2Vec's contextual consistency and hierarchical loss.
Problem
Learning representations of multivariate time series at timestep granularity that capture temporal structure (trend, periodicity, distribution shifts) and remain robust to missing data, where contrastive temporal tasks give only binary signals and suffer sampling bias on periodic signals.
Approach
Learn time-embeddings jointly with the encoder (Time2Vec-style, normalised to a probability distribution) and use them in pretext tasks: regress the Jensen-Shannon divergence between time-embeddings from representation differences, and predict nearby representations conditioned on the target's time-embedding, giving a continuous rather than binary temporal signal.
Key results
- T-Rep reaches 0.706 average accuracy on 30 UEA datasets, versus 0.699 for TS2Vec and 0.707 for Minirocket, using 128 dimensions.
- On ETT forecasting, T-Rep averages MSE 0.986 and MAE 0.702, against 1.004/0.712 for TS2Vec.
- T-Rep achieves F1 0.757 on Yahoo and 0.666 on Sepsis anomaly detection, versus TS2Vec's 0.733 and 0.619.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T-Rep instead of TS2Vec | Time-series anomaly detection · Yahoo | F1: 0.733 -> 0.757 | Helped |
| Use T-Rep instead of TS2Vec | Time-series anomaly detection · Sepsis | F1: 0.619 -> 0.666 | Helped |
| Use T-Rep instead of Minirocket | UEA time-series classification · 30 UEA datasets | Avg. Acc.: 0.707 -> 0.706 | Hurt |
| Use T-Rep instead of TS2Vec | UEA time-series classification · 30 UEA datasets | Avg. Acc.: 0.699 -> 0.706 | Helped |
| Use T-Rep instead of TS2Vec | ETT forecasting · ETT datasets, averaged across prediction horizons | MSE: 1.004 -> 0.986 | Helped |
| Use T-Rep instead of TS2Vec | ETT forecasting · ETT datasets, averaged across prediction horizons | MAE: 0.712 -> 0.702 | Helped |
| Add time-embedding-conditioned forecasting task (ablation) | T-Rep · ETT forecasting datasets | MSE: 1.022 (+3.7%) -> 0.986 | Helped |
| Add time-embedding-conditioned forecasting task (ablation) | T-Rep · Sepsis | F1: 0.392 (-41%) -> 0.665 | Helped |
| Add time-embedding divergence prediction task (ablation) | T-Rep · ETT forecasting datasets | MSE: 1.003 (+1.7%) -> 0.986 | Helped |
| Add time-embedding divergence prediction task (ablation) | T-Rep · Sepsis | F1: 0.634 (-4.7%) -> 0.665 | Helped |
| Add both new time-embedding pretext tasks (ablation) | T-Rep · ETT forecasting datasets | MSE: 0.999 (+1.3%) -> 0.986 | Helped |
| Add both new time-embedding pretext tasks (ablation) | T-Rep · Sepsis | F1: 0.513 (-22.8%) -> 0.665 | Helped |
Surprising findings
- Minirocket, a method built specifically for time series classification, is the top average-accuracy model on UEA (0.707) slightly above T-Rep (0.706).
- Removing the new pretext tasks lowers forecasting performance below removing only the time-embedding divergence task, indicating the two tasks are interdependent.
Limitations
- No hyperparameter tuning was performed, and pretext task weights were fixed at 0.25 each without optimisation.
- Forecasting uses only ridge regression on frozen representations, and the evaluation excludes covariates.
- Sepsis anomaly detection is supervised on top of representations, so it is not purely self-supervised.
Experimental setup
Datasets: UEA classification archive (30 datasets) UCR archive ETTh1 ETTh2 ETTm1 ETTm2 Yahoo anomaly detection PhysioNet 2019 Sepsis
Benchmarks: UEA classification archive (30 datasets) ETT forecasting (ETTh1, ETTh2, ETTm1, ETTm2) Yahoo anomaly detection PhysioNet 2019 Sepsis UCR DodgerLoopGame and ArticularyWordRecognition (missing data)
Metrics: accuracy MSE MAE F1 score average rank
Baselines: TS2Vec T-Loss TNC TS-TCC DTW Minirocket Informer TCN
Builds on: TS2Vec Time2Vec TCN CPC
Closest papers by meaning
- TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.88)
- Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations (NeurIPS 2024 · similarity 0.88)
- TSLANet: Rethinking Transformers for Time Series Representation Learning (ICML 2024 · similarity 0.87)
- ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting (AAAI 2025 · similarity 0.87)
- Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning (AAAI 2025 · similarity 0.87)
- Structured Temporal Causality for Interpretable Multivariate Time Series Anomaly Detection (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.