Research map / Detection and segmentation / Open-set and few-shot detection
Human Hands As Probes for Interactive Object Understanding
Mohit Goyal, Sahil Modi, Rishabh Goyal, Saurabh Gupta
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add hand appearance and motion consistency to temporal contrastive learning | TSC object features · EPIC-STATES test, all objects, 100% training data | mAP: 84.2 $\pm$ 1.0 -> 84.8 $\pm$ 0.4 | Improved |
| Add hand appearance and motion consistency to temporal contrastive learning | TSC object features · EPIC-STATES test, novel objects, 12.5% training data | mAP: 74.5 $\pm$ 0.9 -> 79.7 $\pm$ 0.6 | Improved |
| Use temporal contrastive learning on object tracks instead of ImageNet features | Object state classification · EPIC-STATES test, all objects, 100% training data | mAP: 83.1 $\pm$ 0.0 -> 84.2 $\pm$ 1.0 | Improved |
| Predict interaction sites from masked surrounding context without segmentation-mask training | ACP · EPIC-ROI test, overall, 0% slack | AP: 55.7 -> 57.4 | Improved |
| Combine ACP predictions with Mask R-CNN | EPIC-ROI prediction · EPIC-ROI test, overall, 0% slack | AP: 66.6 -> 68.7 | Improved |
| Predict grasp affordances using hand-derived supervision | ACP · YCB novel-object test, 7 grasp classes | mAP: 30.2% -> 38.1% | Improved |
| Add temporal self-supervision on EPIC hand tracks to adapt the grasp classifier (ablation) | ACP grasp prediction · YCB novel-object test, 7 grasp classes | mAP: 34.3% -> 38.1% | Improved |
| Add hand appearance to TSC (ablation) | TSC object features · EPIC-STATES validation, novel objects, 12.5% training data | mAP: 72.3 $\pm$ 1.3 -> 73.8 $\pm$ 0.8 | Improved |
| Add hand motion to TSC (ablation) | TSC object features · EPIC-STATES validation, novel objects, 12.5% training data | mAP: 72.3 $\pm$ 1.3 -> 74.6 $\pm$ 1.6 | Improved |
| Add both hand motion and appearance to TSC (ablation) | TSC object features · EPIC-STATES validation, novel objects, 12.5% training data | mAP: 72.3 $\pm$ 1.3 -> 75.1 $\pm$ 0.4 | Improved |
| Use detected object-of-interaction tracks with hand-context tracking rather than no tracking (ablation) | TSC · EPIC-STATES validation | mAP: 79.2 -> 81.2 | Improved |
| Replace predicted object tracks with ground-truth object-of-interaction tracks (ablation) | TSC · EPIC-STATES validation | mAP: 81.2 -> 83.5 | Improved |
Closest papers by meaning
- Analysis of the Hands in Egocentric Vision: A Survey (TPAMI 2020 · similarity 0.90)
- Towards A Richer 2D Understanding of Hands at Scale (NeurIPS 2023 · similarity 0.90)
- Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning (ICLR 2025 · similarity 0.89)
- Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions? (ICLR 2025 · similarity 0.88)
- GEARS: Local Geometry-aware Hand-object Interaction Synthesis (CVPR 2024 · similarity 0.88)
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
