Research map / Video / Video segmentation and tracking
MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoPFormer, a self-supervised Transformer that tokenizes IMU signals into VQ-learned motion primitives with metadata-aware embeddings, pre-trained with masked motion modeling, for interpretable and cross-dataset wearable HAR.
Problem
Wearable-sensor human activity recognition suffers from limited interpretability and poor cross-dataset generalization due to data heterogeneity (sensor types, placement, sampling rates).
Approach
Discretize IMU segments into a learned VQ codebook of "motion primitives" (analogous to words), add per-channel sensor metadata embeddings from a pretrained text embedding model, and pretrain a Transformer with masked prediction of primitive indices so the primitives can be inspected for interpretability.
Key results
- MoPFormer reaches 90.17% average accuracy and 89.23% average F1 across six HAR datasets, the highest among compared methods (TSLANet 89.28% acc, 88.13% F1 average is higher in accuracy, so MoPFormer leads on F1 only in the main table's average-F1 column).
- On MHealth MoPFormer achieves 93.22% accuracy and 92.28% F1, versus 89.84% for TSLANet.
- Removing pretraining drops UCI-HAR accuracy from 90.58% to 77.58%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pretraining to MoPFormer (ablation) | MoPFormer · PAMAP2 | accuracy: 85.76 -> 86.08 | Helped |
| Add pretraining to MoPFormer (ablation) | MoPFormer · DSADS | accuracy: 94.68 -> 97.60 | Helped |
| Add statistical features (ablation) | MoPFormer · PAMAP2 | accuracy: 69.95 -> 86.08 | Helped |
| Add statistical features (ablation) | MoPFormer · DSADS | accuracy: 71.15 -> 97.60 | Helped |
| Add metadata embedding (ablation) | MoPFormer · PAMAP2 | accuracy: 58.18 -> 86.08 | Helped |
| Add metadata embedding (ablation) | MoPFormer · DSADS | accuracy: 80.94 -> 97.60 | Helped |
| Use motion primitive length 100 instead of 50 (ablation) | MoPFormer · PAMAP2 | accuracy: 86.08 -> 80.81 | Hurt |
| Use motion primitive length 100 instead of 50 (ablation) | MoPFormer · DSADS | accuracy: 97.60 -> 92.94 | Hurt |
| Use 2 Transformer layers instead of 5 (ablation) | MoPFormer · PAMAP2 | accuracy: 86.08 -> 85.01 | Hurt |
| Use 2 Transformer layers instead of 5 (ablation) | MoPFormer · DSADS | accuracy: 97.60 -> 96.92 | Hurt |
| Increase Transformer depth from 5 to 7 or 10 layers (ablation) | MoPFormer · PAMAP2 and DSADS | accuracy: 86.08, 86.11, 86.05; 97.60, 97.53, 97.28 | No clear effect |
| Reduce hidden embedding size to 32 or 64 (ablation) | MoPFormer · PAMAP2 and DSADS | accuracy: 82.18, 85.90; 94.81, 96.93 | Hurt |
Surprising findings
- Removing statistical features drops PAMAP2 accuracy to 69.95%, and removing metadata embedding drops it to 58.18%, far more than removing pretraining.
- Multi-run re-evaluation on PAMAP2 gives 87.49±4.10% accuracy, above the originally reported 86.08%, with large variance.
Limitations
- Computational overhead from vector quantization and multi-layer Transformer may hinder real-time inference on resource-limited wearables.
- Pretraining on five datasets and evaluation on the sixth; the fine-tuning protocol freezes the encoder and trains only a classifier head, and multi-seed results show high variance on some datasets.
- Interpretability analysis is qualitative (similarity matrices, frequency plots) without quantitative validation of semantic meaning.
Experimental setup
Datasets: PAMAP2 DSADS MHealth Realworld UCI-HAR USC-HAD
Benchmarks: PAMAP2 DSADS MHealth Realworld UCI-HAR USC-HAD
Metrics: accuracy macro-F1
Baselines: BYOL+perm_jit BYOL+lfc BYOL+Mixup ModCL TSLANet CALANet w/o Pretrain
Builds on: VQ-VAE BYOL TSLANet CALANet
Closest papers by meaning
- Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition - And Ways to Overcome Them (AAAI 2025 · similarity 0.87)
- Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition (AAAI 2025 · similarity 0.86)
- Inertia-1: An Open Exploration of Wearable Motion Foundation Models (NeurIPS 2026 · similarity 0.86)
- ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition (AAAI 2025 · similarity 0.85)
- RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data (ICLR 2025 · similarity 0.85)
- Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.