Research map / Video / Action recognition
Count What You Want: Exemplar Identification and Few-Shot Counting of Human Actions in the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an exemplar-based few-shot counter for wearable sensor data where users specify actions by saying "one, two, three", plus a distance-preserving loss and synthetic pretraining, and releases the DWC dataset.
Problem
Counting human actions of interest (including infrequent, non-dominant actions) from wearable sensor data, where class-specific and class-agnostic counters fall short for arbitrary classes.
Approach
Users provide exemplars by vocally counting "one, two, three"; the method detects these utterances in audio with a dynamic-programming constrained search, builds exemplar-to-sequence similarity maps (correlation combined with Soft-DTW), fuses them into embeddings, and estimates density whose sum gives the count. A distance-preserving Laplacian loss and template-based synthetic pretraining further
Key results
- On the DWC test set the proposed method achieves MAE 7.47 and RMSE 13.09, versus 10.82 MAE for RepNet and 12.97 for TransRAC.
- Ablation on the validation set shows synthetic pretraining has the largest effect, reducing MAE from 10.05 to 7.66 when added to the full model.
- Using three audio-specified exemplars gives MAE 7.66 on validation versus 9.08 with one exemplar.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add similarity estimation (ablation) | DWC counting method · DWC validation set | MAE: 11.30 -> 10.87 | Helped |
| add constrained detection of exemplar utterances (ablation) | DWC counting method with similarity estimation · DWC validation set | MAE: 10.87 -> 10.32 | Helped |
| add distance-preserving loss (ablation) | DWC counting method with similarity estimation and constrained detection · DWC validation set | MAE: 10.32 -> 10.05 | Helped |
| pretrain on synthesized data (ablation) | DWC counting method with similarity estimation, constrained detection, and distance-preserving loss · DWC validation set | MAE: 10.05 -> 7.66 | Helped |
| increase audio exemplars from one to two (ablation) | Proposed DWC counting method · DWC validation set | MAE: 9.08 -> 8.74 | Helped |
| increase audio exemplars from two to three (ablation) | Proposed DWC counting method · DWC validation set | MAE: 8.74 -> 7.66 | Helped |
| use synthesized pretraining data twice the size of the real training set (ablation) | Proposed DWC counting method · DWC validation set | counting performance | Helped |
Limitations
- Requires the user to vocalize 'one, two, three' at the start of each counting session.
- Evaluated only on a single custom wearable dataset collected with one watch model (Samsung Galaxy Watch 4).
- Some data collected with timing mismatches between audio and sensor streams.
Experimental setup
Datasets: DWC Speech Command
Benchmarks: DWC
Metrics: MAE RMSE OBK
Baselines: Mean Frequency-based RepNet TransRAC
Builds on: RepNet TransRAC Soft-DTW BC_ResNet Laplacian Eigenmaps
Closest papers by meaning
- CountLLM: Towards Generalizable Repetitive Action Counting via Large Language Model (CVPR 2025 · similarity 0.84)
- SensorLM: Learning the Language of Wearable Sensors (NeurIPS 2025 · similarity 0.83)
- UniMTS: Unified Pre-training for Motion Time Series (NeurIPS 2024 · similarity 0.83)
- ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition (NeurIPS 2024 · similarity 0.83)
- DreamCatcher: A Wearer-aware Multi-modal Sleep Event Dataset Based on Earables in Non-restrictive Environments (NeurIPS 2024 · similarity 0.82)
- MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.