Research map / 3D vision / Human pose estimation
Real-Time Multimodal Fingertip Contact Detection via Depth and Motion Fusion for Vision-Based Human-Computer Interaction
Mukhiddin Toshpulatov, Wookey Lee, Suan Lee, Geehyuk Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune pretrained depth model on the fingertip-surface dataset | Depth Anything V2 · Fingertip-surface test set | MAE (mm): 12.3 -> 3.8 | Improved |
| fine-tune depth models on target data instead of unsupervised domain adaptation (AdaBN, DANN, MMD) | monocular depth models · Fingertip-surface MAE | MAE (mm): 8.7-9.2 mm -> 3.8 mm | Improved |
| add depth estimation to contact detection (ablation) | Full system · Fingertip-surface test set | F1 (%): 86.4 -> 94.4 | Improved |
| add velocity gate to contact detection (ablation) | Full system · Fingertip-surface test set | F1 (%): 88.7 -> 94.4 | Improved |
| add 450ms cooldown to contact detection (ablation) | Full system · Fingertip-surface test set | F1 (%): 93.6 -> 94.4 F1; false positive rate 3.1 -> 4.2 | Mixed |
| add velocity gate on top of depth fine-tuning (ablation) | Typing system · Typing WPM | WPM: 35.7 -> 45.6 | Improved |
| add depth fine-tuning without velocity gate (ablation) | Typing system · Typing CER | CER (%): 12.4 -> 8.7 | Improved |
| add state machine with hysteresis (ablation) | Typing system · Typing CER | CER (%): 10.5 -> 3.1 | Improved |
| add depth fusion (ablation) | Typing system · Typing WPM | WPM: 31.2 -> 43.8 | Improved |
| add cooldown (ablation) | Typing system · Typing WPM | WPM: 38.0 -> 43.8 | Improved |
| raise input resolution from 640x480 to 848x480 | Typing system · Typing WPM | WPM: 43.8 -> 45.6 | Improved |
| use semi-reflective laminate surface instead of white desk | Full system · Cross-surface robustness | F1 (%): 85.3 (D405), 78.2 (RGB) | Hurt |
Closest papers by meaning
- MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generation (NeurIPS 2025 · similarity 0.86)
- Learning Dense Hand Contact Estimation from Imbalanced Data (NeurIPS 2025 · similarity 0.85)
- GEARS: Local Geometry-aware Hand-object Interaction Synthesis (CVPR 2024 · similarity 0.85)
- QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects (AAAI 2025 · similarity 0.85)
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · similarity 0.84)
- How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
