Research map / Time series / Foundation models for time series

OTIS: Learning High-Quality Time Series Features With Tiny Encoders

NeurIPS 2026 · Time series › Foundation models for time series · approach: LLM-based time series forecasting · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OTIS, a 7.1M-parameter time series encoder pre-trained with masked modelling plus domain-aware tokenisation, dual masking, and an NCC loss, matching much larger encoders across 162 tasks.

Problem

General-purpose time series encoders rely on large scale to handle heterogeneous multi-domain data, which is infeasible for resource-constrained wearables and industrial sensors.

Approach

Tailor standard masked data modelling to time series via domain-specific variate embeddings (domain-aware tokeniser), alternating random and post-fix (causal) masking, and a normalised cross-correlation loss added to MSE, letting a tiny encoder match 54x larger encoders.

Key results

• OTIS-7.1M wins 16 of 23 UEA datasets in 5-shot classification (55.7% vs MOMENT-386M 51.5%) with about 6x less memory and 45x less energy than MOMENT.
• OTIS achieves 0.614 balanced accuracy on TUEV event type versus 0.517 for MOMENT-386M, and best R² on all five UK BioBank ECG regression targets.
• Ablation: removing the NCC loss causes the largest drop in forecasting (MSE 0.976 to 1.161), and removing pre-training lowers classification accuracy from 85.7 to 75.1.

Surprising findings

• Larger OTIS variants (40M and 116M) offer almost no advantage over the 7.1M base, so performance saturates with encoder size.
• Frozen-encoder UCR 5-shot results for OTIS (59.8%) remain below MOMENT-386M (60.9%), so the tiny model does not win across all archives.

Limitations

• Performance saturates with encoder size, so scaling beyond 7.1M requires further methodological advances.
• Relies on manually curated pre-training data; automated web-scale curation is not addressed.
• Evaluation is restricted to regularly sampled continuous time series, excluding irregular sampling.

Experimental setup

Datasets: MIMIC-IV-ECG DWD AudioSet-20K FD-A TDBrain SEED NN5 FRED-MD Exchange FD-B Gesture EMG

Benchmarks: FD-B Gesture EMG UEA UCR Epilepsy SleepEDF TUEV UK BioBank ETTh1 ETTh2 ETTm1

Metrics: accuracy balanced accuracy R-squared MSE NCC memory energy latency throughput

Baselines: MOMENT-386M UniTS-8.2M TS2Vec SimMTM CBraMod CSBrain MMCL Time-MoE

Builds on: masked data modelling Transformer AdamW DINO

Closest papers by meaning

• TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.86)
• Rotary Masked Autoencoders are Versatile Learners (NeurIPS 2025 · similarity 0.85)
• Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification (AAAI 2025 · similarity 0.85)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.85)
• Time Series Representations with Hard-Coded Invariances (ICML 2025 · similarity 0.85)
• Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.84)