Research map / Time series / Foundation models for time series
UniMTS: Unified Pre-training for Motion Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains a spatio-temporal graph encoder contrastively on physics-simulated IMU data (from HumanML3D skeletons) paired with LLM-paraphrased text, with rotation-invariant augmentation, enabling zero-shot, few-shot and full-shot motion time series classification across 18 real datasets.
Problem
Motion time series classifiers from wearables generalize poorly across device placement, device orientation, and activity types, and large-scale labeled motion time series data are scarce for pre-training.
Approach
Synthesize accelerations and angular velocities for every skeleton joint from motion-capture skeleton data via a physics engine, align them with LLM-augmented text descriptions via CLIP-style contrastive learning, encode with a spatio-temporal graph network with joint masking, and apply random rotation augmentation for orientation invariance.
Key results
- Zero-shot average accuracy 43.5% vs 12.5% for best baseline ImageBind (reported as 340% relative improvement).
- Full-shot average accuracy 90.6% vs 85.8% for best baseline BioBankSSL/SHARE (reported as 9.2% improvement).
- Ablations show removing rotation augmentation drops zero-shot average accuracy from 43.5% to 27.1%, and removing graph encoder to 25.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add rotation-invariant augmentation (ablation) | UniMTS · 18 motion time-series datasets, zero-shot | accuracy: 27.1 -> 43.5 | Helped |
| add text augmentation (ablation) | UniMTS · 18 motion time-series datasets, zero-shot | accuracy: 39.8 -> 43.5 | Helped |
| use a graph encoder instead of a CNN-based encoder (ablation) | UniMTS · 18 motion time-series datasets, zero-shot | accuracy: 25.1 -> 43.5 | Helped |
| pre-train instead of using random initialization (ablation) | UniMTS · 18 motion time-series datasets, full-shot | accuracy: 84.7 -> 90.6 | Helped |
| pre-train instead of using random initialization (ablation) | UniMTS · 18 motion time-series datasets, full-shot | F1: 78.5 -> 87.5 | Helped |
| add KIT-ML and NTU RGB+D 120 to the pre-training corpus (ablation) | UniMTS · pre-training corpus | performance | No clear effect |
Surprising findings
- Zero-shot UniMTS does worse than the w/o text augmentation variant on several datasets, and the w/o text aug model reaches 39.8% average accuracy, showing text augmentation is not uniformly beneficial.
- Pre-trained UniMTS is only trained on simulated data yet its t-SNE embeddings of real data form semantically aligned clusters.
Limitations
- Simulated signals only approximate real sensor data collected near rather than on joints; random offset vectors are planned but not implemented.
- Evaluated only on classification; other motion tasks such as inertial navigation are not covered.
- Pre-training relies on a single motion skeleton corpus (HumanML3D); adding KIT-ML and NTU RGB+D did not help.
Experimental setup
Datasets: HumanML3D Opportunity UCI-HAR MotionSense w-HAR Shoaib HAR70+ RealWorld TNDA-HAR PAMAP2 USC-HAD Mhealth
Benchmarks: Opportunity UCI-HAR MotionSense w-HAR Shoaib HAR70+ RealWorld TNDA-HAR PAMAP2 USC-HAD Mhealth Harth
Metrics: accuracy macro-F1 R@2
Baselines: ImageBind IMU2CLIP IMUGPT HARGPT LLaVA GPT4TS BioBankSSL SHARE
Builds on: CLIP ST-GCN ImageBind IMU2CLIP
Closest papers by meaning
- Inertia-1: An Open Exploration of Wearable Motion Foundation Models (NeurIPS 2026 · similarity 0.87)
- Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions (ICML 2025 · similarity 0.85)
- RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data (ICLR 2025 · similarity 0.85)
- ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition (AAAI 2025 · similarity 0.85)
- MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition (NeurIPS 2025 · similarity 0.84)
- Scaling Wearable Foundation Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.