Research map / Video / Video segmentation and tracking
Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DecomposeWHAR, which decomposes multi-sensor IMU signals per sensor and variable with depthwise/pointwise convolutions, then fuses them with an SSM (Mamba) for temporal aggregation and self-attention across sensors for inter-sensor interaction.
Problem
Wearable human activity recognition from multi-sensor IMU data needs to capture spatio-temporal relationships among intra-sensor variables and inter-sensor correlations, which shared convolutions, dense/RNN fusion, and GCN-based sensor graphs handle poorly.
Approach
Decompose signals at sensor, temporal, variable and channel levels using depthwise-separable convolution with per-variable groups, then fuse hierarchically (channel, variable, global temporal via Mamba, then cross-sensor self-attention) instead of GCN-based sensor graphs.
Key results
- On Opportunity, DecomposeWHAR reaches 78.28% accuracy and 72.04% macro-F1, versus 75.13% and 67.05% for the best baseline HARMamba.
- On Realdisp, DecomposeWHAR reaches 96.64% accuracy and 96.10% macro-F1, while on Skoda it reaches 97.61% accuracy and 97.24% macro-F1, beating DynamicWHAR by about 4%.
- Ablation shows removing the decomposition phase drops Opportunity macro-F1 by 3.07 points and Skoda by 4.05 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the Decomposition Phase instead of using Conv2D (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$3.07); ($-$4.07); ($-$4.05) | Helped |
| add GTA (Mamba) (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$2.71); ($-$5.82); ($-$11.09) | Helped |
| add CSI (Attention) (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$3.50); ($-$5.25); ($-$18.00) | Helped |
| add both GTA and CSI (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$9.27); ($-$10.56); ($-$19.51) | Helped |
| use GTA rather than Attention for global temporal aggregation (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$1.78); ($-$0.93); ($-$10.62) | Helped |
| place GTA before CSI rather than after CSI (ablation) | DecomposeWHAR · Opportunity, Realdisp, and Skoda | Macro-F1: ($-$0.64); ($-$1.81); ($-$1.87) | Helped |
| replace CSI with a GCN (ablation) | DecomposeWHAR · Opportunity | Macro-F1 | Hurt |
| set MSE patch size P to 1/3 of the total time steps L (ablation) | DecomposeWHAR · Opportunity | performance | Helped |
| set MSE output channels D to 64 (ablation) | DecomposeWHAR · Opportunity | performance | Helped |
| set LTE kernel size to 3 (ablation) | DecomposeWHAR · Opportunity | performance | Helped |
Surprising findings
- On Realdisp, the HARMamba baseline (91.29% accuracy) trails DynamicWHAR (92.58%), and the 'Attention as GTA' ablation remains close to the full model on Realdisp (95.17% vs 96.10% macro-F1).
Limitations
- Mamba block has yet to be refined; parameter size and computational cost could be reduced.
- FLOPs comparisons were not shown in the figure due to significant discrepancies.
- Skoda evaluated with a single hold-out split rather than leave-one-subject-out.
Experimental setup
Datasets: Opportunity Realdisp Skoda
Benchmarks: Opportunity Realdisp Skoda
Metrics: accuracy macro-F1 FLOPs parameters inference latency energy consumption
Baselines: DeepConvLSTM Attend and Discriminate GraphConvLSTM HAR-PBD DynamicWHAR IF-ConvTransformer HARMamba Att. Model
Builds on: Depthwise Separable Convolution Mamba DynamicWHAR Attend and Discriminate HARMamba
Closest papers by meaning
- MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition (NeurIPS 2025 · similarity 0.86)
- ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition (AAAI 2025 · similarity 0.85)
- Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition - And Ways to Overcome Them (AAAI 2025 · similarity 0.84)
- HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors (AAAI 2024 · similarity 0.84)
- Inertia-1: An Open Exploration of Wearable Motion Foundation Models (NeurIPS 2026 · similarity 0.83)
- Scalable Hierarchical Deep Neural Network for Time Series Analysis in Wearable Sensor-based Human Activity Recognition (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.