Research map / Label-efficient and robust learning / Recognition and long-tail
emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyography
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces emg2qwerty, a large public dataset of wrist sEMG recordings from 108 users typing on a QWERTY keyboard, plus ASR-style CTC baselines for generic and personalized key-press transcription.
Problem
Non-invasive wrist sEMG as a high-bandwidth human-computer text input lacks large-scale public datasets with well-defined tasks and benchmarks; sEMG signals show strong domain shift across users and sessions.
Approach
Frame touch-typing key-press recognition from wrist sEMG as an ASR-like sequence transduction problem, borrowing TDS convolutions, CTC loss, SpecAugment, and a character n-gram LM with a backspace-aware beam search, plus band-rotation and temporal-jitter augmentations and user-level generalization/personalization benchmarks.
Key results
- Personalized finetuned model with 6-gram char LM achieves mean test CER of 6.95% across 8 held-out users (best user 3.16%).
- Generic model without personalization yields over 50% CER on unseen users (51.78% with LM), indicating strong domain shift.
- Finetuning from the generic model outperforms random-init personalized models (test CER 11.28% vs 15.38% without LM).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Personalize using user-specific sessions instead of using the generic model without personalization | Generic user model · Held-out test users | CER: 55.38 \pm 4.10 -> 15.38 \pm 5.88 | Helped |
| Initialize the personalized model with generic-model weights and finetune on user-specific sessions instead of random initialization (ablation) | Personalized model · Held-out test users | CER: 15.38 \pm 5.88 -> 11.28 \pm 4.45 | Helped |
| Add a 6-gram character-level language model (ablation) | Generic user model · Held-out test users | CER: 55.38 \pm 4.10 -> 51.78 \pm 4.61 | Helped |
| Add a 6-gram character-level language model (ablation) | Personalized model (random-init) · Held-out test users | CER: 15.38 \pm 5.88 -> 9.55 \pm 5.16 | Helped |
| Add a 6-gram character-level language model (ablation) | Personalized model (finetuned) · Held-out test users | CER: 11.28 \pm 4.45 -> 6.95 \pm 3.61 | Helped |
| Use spectral features instead of rectifying the time-domain signal (ablation) | emg2qwerty baseline · emg2qwerty | CER | Helped |
| Add SpecAugment (ablation) | emg2qwerty baseline · emg2qwerty | CER | Helped |
| Use CTC loss instead of cross-entropy classification loss (ablation) | emg2qwerty baseline · emg2qwerty | CER | Helped |
Surprising findings
- The generic model trained on 100 users is essentially unusable on unseen users (CER above 50%), showing severe cross-user domain shift.
Limitations
- Requires touch-typing skill on a QWERTY keyboard and supports only English.
- Data collected on a physical keyboard at a desk, not keyboard-free typing, causing possible domain mismatch.
- Deployment on wristband hardware (latency, compute, battery) not addressed; models run offline.
Experimental setup
Datasets: emg2qwerty WikiText-103
Benchmarks: emg2qwerty
Metrics: character error rate (CER)
Baselines: Generic (no personalization) model Personalized (random-init) model
Builds on: Time-Depth Separable ConvNets SpecAugment Connectionist Temporal Classification KenLM
Closest papers by meaning
- SplashNet: Split‑and‑Share Encoders for Accurate and Efficient Typing with Surface Electromyography (NeurIPS 2025 · similarity 0.91)
- emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation (NeurIPS 2024 · similarity 0.91)
- SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network (NeurIPS 2024 · similarity 0.89)
- EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyography (NeurIPS 2024 · similarity 0.88)
- Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in sEMG Analysis (ICML 2025 · similarity 0.87)
- Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.