Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition

AAAI 2025 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Generative feature synthesis for zero-shot · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ZeroHAR, a two-stage zero-shot wearable HAR framework that aligns IMU time series with textual sensor context (sensor type, axis, body position) via contrastive learning, then trains action recognition against LLM-generated activity descriptions.

Problem

Zero-shot recognition of human activities from wearable IMU sensor data, where models trained on a limited set of activities must generalize to unseen activities across varying sensors, placements, and datasets.

Approach

Augments IMU data with sensor context text (sensor type, axis, body position) and aligns the IMU latent space with it through multimodal InfoNCE contrastive learning before training on LLM-generated biomechanical activity descriptions, using a frozen ImageBind text encoder as the sensor-context embedder.

Key results

• ZeroHAR averages 45.6% zero-shot accuracy and 35.9 macro-F1 across 18 datasets, a 262% relative accuracy improvement over the second-best baseline (HARGPT, 12.6%).
• Macro-F1 improves 392% over the second-best result (ImageBind, 7.3).
• Stage I pre-training speeds Stage II convergence and lowers its loss on Opportunity; training Stage I and II jointly (multitask) yields 33.7% accuracy versus 45.6% for sequential training.

Measured effects reported in the paper

Surprising findings

• Stage II alone (without sensor-context alignment) performs poorly at 9.7% average accuracy, showing sensor-context alignment is essential.
• IMUGPT scores 64.2% accuracy on w-HAR but near 0% on several other datasets, indicating unstable cross-dataset generalization.

Limitations

• Activity descriptions are generated by GPT-4 and manually checked only for accuracy, which may introduce bias or errors.
• Zero-shot accuracy drops sharply on hard datasets with many activities (e.g., MMAct 8.6%, UTD-MHAD 18.7%).
• Relies on the frozen ImageBind text encoder, whose IMU alignment is limited to specific sensors and body positions.

Experimental setup

Datasets: Opportunity UCI-HAR MotionSense w-HAR Shoaib HAR70+ RealWorld TNDA-HAR PAMAP2 USC-HAD Mhealth Harth

Benchmarks: Opportunity UCI-HAR MotionSense w-HAR Shoaib HAR70+ RealWorld TNDA-HAR PAMAP2 USC-HAD Mhealth Harth

Metrics: accuracy macro-F1

Baselines: NuActiv SemAtt NCBM LETS-GZSL SemHAR SHARE NonViz ImageBind IMU2CLIP IMUGPT HARGPT LLaVA

Builds on: ImageBind InfoNCE Cross-Modal Contrastive Multiview Coding ZSL attribute-based HAR

Closest papers by meaning

• Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition (AAAI 2025 · similarity 0.85)
• SensorLM: Learning the Language of Wearable Sensors (NeurIPS 2025 · similarity 0.85)
• MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition (NeurIPS 2025 · similarity 0.85)
• UniMTS: Unified Pre-training for Motion Time Series (NeurIPS 2024 · similarity 0.85)
• Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition (CVPR 2024 · similarity 0.84)
• Inertia-1: An Open Exploration of Wearable Motion Foundation Models (NeurIPS 2026 · similarity 0.83)