Research map / Time series / Foundation models for time series
Time Series Representations with Hard-Coded Invariances
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes hard-coded invariant convolutions for time series that are exactly invariant to offset shift, amplitude scaling and linear trend, and combines them with standard convolutions in a single-layer InvConvNet.
Problem
Standard convolutional networks are sensitive to common time series deformations such as amplitude scaling, offset shifts and trends, which hurts robustness, generalization and classification performance.
Approach
Model deformations as a group action (rigid deformations R*+ ⋉ H) and build an orbit-injective invariant embedding via projection onto the orthogonal complement of the deformation subspace; use it to define exact invariant convolutions computed efficiently with FFT, and combine standard, offset-invariant and trend-invariant filters in one inception-like layer.
Key results
- InvConvNet reaches 71.81% average accuracy on 26 UEA datasets, versus 71.29% for ROCKET and 66.87% for TimesNet, with a statistically tied mean rank to ROCKET.
- Under a smooth random walk trend deformation, the trend-invariant convolution drops about 6% on average versus 59% for standard convolution and 34% for TS-TCC.
- On fault-diagnosis transfer learning, supervised InvConvNet averages 76.52% versus 72.18% for TS2Vec and 70.76% for TS-TCC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine standard, offset-invariant, and trend-invariant convolutions (ablation) | INVCONVNET · UEA (26 datasets) | accuracy: 68.04 ± 1.76; 67.70 ± 1.21; 66.61 ± 2.58 -> 71.81 ± 0.80 | Helped |
| Combine standard, offset-invariant, and trend-invariant convolutions (ablation) | INVCONVNET · UCIHAR | accuracy: 96.13 ± 0.34; 95.75 ± 0.51; 95.43 ± 0.13 -> 96.63 ± 0.49 | Helped |
| Combine standard, offset-invariant, and trend-invariant convolutions (ablation) | INVCONVNET · Sleep-EDF | accuracy: 84.70 ± 0.39; 83.73 ± 0.13; 83.14 ± 0.13 -> 84.95 ± 0.39 | Helped |
| Combine standard, offset-invariant, and trend-invariant convolutions (ablation) | INVCONVNET · Epilepsy | accuracy: 98.09 ± 0.13; 98.26 ± 0.04; 98.25 ± 0.09 -> 98.43 ± 0.04 | Helped |
| Use trend-invariant instead of standard convolutions | convolutional classifier · HandOutlines, + off., LT | accuracy: 35.68 ± 0.47 -> 70.63 ± 0.41 | Helped |
| Use trend-invariant instead of standard convolutions | convolutional classifier · NonInvasiveFetalECGThorax1, + off., LT | accuracy: 6.09 ± 0.17 -> 82.02 ± 0.52 | Helped |
| Use a pool of standard, offset-invariant, and trend-invariant convolutions | INVCONVNET · Sleep-EDF | accuracy: 84.95 ± 0.39 vs 85.62 ± 0.13 | Mixed |
| Use invariant convolutions rather than standard-only convolutions | INVCONVNET · Fault-Diagnosis transfer (average) | accuracy: 72.66 ± 0.78 -> 76.52 ± 0.37 | Helped |
Surprising findings
- Standard convolutions lose up to 48-59% accuracy under small offset or trend deformations, while the trend-invariant convolution stays nearly constant.
- Invariant convolutions with a single layer are competitive with deep architectures; the method is faster and lighter than most baselines (training 1.90 s/epoch, 199.4K parameters on Heartbeat).
Limitations
- Only affine (offset, linear trend, scaling) deformations are modeled; non-linear trends and seasonal components are left for future work.
- Fixed pooling of invariant and standard filters; alternative pooling such as attention or RL is not explored.
- Extension to unsupervised learning is not yet done.
Experimental setup
Datasets: UEA (26 datasets) UCIHAR Sleep-EDF Epilepsy Fault-Diagnosis UCR archive (5 datasets) MIT-BIH
Benchmarks: UEA (26 datasets) UCIHAR Sleep-EDF Epilepsy UCR deformation robustness (5 datasets) Fault-Diagnosis transfer learning
Metrics: classification accuracy percentage drop in accuracy training time per epoch model size
Baselines: TimesNet PatchTST Crossformer DLinear iTransformer TimeMixer Peri-MidFormer TSLANet Inception ResNet CNN ROCKET
Builds on: Inception-Time TS-TCC ROCKET Z-normalization LT-normalized distance
Closest papers by meaning
- TVNet: A Novel Time Series Analysis Method Based on Dynamic Convolution and 3D-Variation (ICLR 2025 · similarity 0.86)
- T-Rep: Representation Learning for Time Series using Time-Embeddings (ICLR 2024 · similarity 0.86)
- TSLANet: Rethinking Transformers for Time Series Representation Learning (ICML 2024 · similarity 0.86)
- Abstracted Shapes as Tokens - A Generalizable and Interpretable Model for Time-series Classification (NeurIPS 2024 · similarity 0.86)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.85)
- TimePoint: Accelerated Time Series Alignment via Self-Supervised Keypoint and Descriptor Learning (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.