Research map / Video / Video segmentation and tracking
Accessible, At-Home Detection of Parkinson’s Disease via Multi-Task Video Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a 1102-session webcam video dataset (finger tapping, smile, pangram speech) for Parkinson's screening and UFNet, an uncertainty-calibrated multi-task fusion network using MC dropout and attention, reaching 87.3% accuracy and 92.8% AUROC.
Problem
At-home, accessible detection of Parkinson's disease is limited by unimodal analysis of motor or speech tasks, and there is no large multi-task video dataset for PD screening.
Approach
Train independent task-specific shallow networks with MC dropout to obtain per-task predictions and uncertainties, then fuse task features and predictions with self-attention whose scores are down-weighted by task-specific uncertainty, and withhold predictions when uncertain.
Key results
- UFNet with all three tasks achieved 87.3 ± 0.4% accuracy, 92.8 ± 0.2% AUROC and 81.0 ± 0.6% F1 on the test set, outperforming majority voting and fusion baselines.
- With MC dropout 95% CI withholding, UFNet reached 88.0 ± 0.3% accuracy and 93.0 ± 0.2% AUROC while predicting on 97.8% of data.
- Speech alone was the strongest single task (84.5% accuracy, 89.4% AUROC), while finger-tapping was weakest (72.0% accuracy).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse all three task modalities with UFNet instead of using a single task | UFNet · PD test set | AUROC: 74.9 ± 0.7 -> 92.8 ± 0.2 | Helped |
| Use UFNet instead of the strongest multimodal fusion baseline | PD test set · PD test set | accuracy: 84.1 ± 0.3 -> 87.3 ± 0.4 | Helped |
| Replace calibrated self-attention with dot product self-attention (ablation) | UFNet · PD test set | AUROC: 92.8 ± 0.2 -> 92.9 ± 0.2 | No clear effect |
| Use calibrated self-attention instead of dot product self-attention (ablation) | UFNet · PD test set | accuracy: 85.5 ± 0.4 -> 87.3 ± 0.4 | Helped |
| Replace calibrated self-attention with LRFormer (ablation) | UFNet · PD test set | accuracy: 86.2 ± 0.5 -> 87.3 ± 0.4 | Helped |
| Add task-specific prediction scores to early fusion (hybrid fusion) (ablation) | UFNet · PD test set | F1 score: 79.9 ± 0.8 -> 81.0 ± 0.6 | Helped |
| Add MC dropout (ablation) | Smile model · Smile | accuracy: 75.6 ± 0.2 -> 77.6 ± 0.2 | Helped |
| Add MC dropout (ablation) | Speech model · Speech | AUROC: 89.4 ± 0.2 -> 87.8 ± 0.1 | Hurt |
| Add MC dropout (ablation) | Finger-tapping model · Finger-tapping | accuracy: 72.0 ± 0.9 -> 73.1 ± 0.7 | No clear effect |
| Use both hands' tapping features instead of left-hand-only features (ablation) | Finger-tapping model · Finger-tapping | F1 score: 52.6 -> 60.2 | Helped |
| Use MC Dropout + 95% CI to withhold uncertain predictions | UFNet hybrid fusion · Withholding predictions | accuracy: 87.3 ± 0.4 -> 88.0 ± 0.3 | Helped |
| Use conformal prediction rather than MC Dropout + 95% CI for withholding (ablation) | UFNet hybrid fusion · Withholding predictions | accuracy: 88.0 ± 0.3 -> 93.9 ± 0.7 | Helped |
Surprising findings
- Finger-tapping was the weakest single task, yet adding its features improved multi-task performance in most metrics.
- The UFNet with Smile + Speech reached 88.76% accuracy on YouTubePD but with lower F1 (58.33%) and AUROC (83.83%) than the prior state of the art.
Limitations
- Raw videos cannot be shared; only extracted features are released.
- Accuracy drops for participants aged 30-49 and over 80 due to under-representation; recommended only for ages 50-80.
- Speech task requires English pangram, limiting use for non-English speakers; recall is 78.4%, so some PD cases are missed.
Experimental setup
Datasets: Webcam PD multi-task video dataset (finger-tapping, smile, speech) YouTubePD
Benchmarks: multi-task PD webcam video dataset YouTubePD
Metrics: accuracy AUROC F1 score sensitivity specificity precision balanced accuracy expected calibration error Brier score coverage
Baselines: Majority Voting Neural Late Fusion Early Fusion Baseline Hybrid Fusion Baseline Dot product self-attention LRFormer YouTubePD ViViT TimeSformer VideoMAE Uniformer
Builds on: Monte Carlo dropout LRFormer WavLM MediaPipe
Closest papers by meaning
- TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease (CVPR 2024 · similarity 0.89)
- Care-PD: A Multi-Site Anonymized Clinical Dataset for Parkinson’s Disease Gait Assessment (NeurIPS 2025 · similarity 0.87)
- Breaking Data Silos in Parkinson’s Disease Diagnosis: An Adaptive Federated Learning Approach for Privacy-Preserving Facial Expression Analysis (AAAI 2025 · similarity 0.86)
- DECT: Harnessing LLM-assisted Fine-Grained Linguistic Knowledge and Label-Switched and Label-Preserved Data Generation for Diagnosis of Alzheimer’s Disease (AAAI 2025 · similarity 0.81)
- CLIMB: Data Foundations for Large Scale Multimodal Clinical Foundation Models (ICML 2025 · similarity 0.81)
- Neurodegenerative Brain Network Classification via Adaptive Diffusion with Temporal Regularization (ICML 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.