AI Paper Atlas

Research map / 3D vision / Human pose estimation

Transformer-Based Unified Recognition of Two Hands Manipulating Objects

Hoseong Cho, Chanwoo Kim, Jihyeon Kim, Seongyeong Lee, Elkhan Ismayilzada, Seungryul Baek

CVPR 2023 · 3D vision › Human pose estimation · approach: Temporal video pose estimation · 21 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reference point refinement to the hand-object pose estimator (ablation)fHOP pose estimator · H2O and FPHA testmean end-point error (mm): Left hand -3.5mm, right hand -5.3mm, object -5.8mm on H2O; right hand -9.8mm, object -19.8mm on FPHAImproved
add contact map to mesh vertex input of the interaction recognizer (ablation)fIA interaction recognizer · H2O testtop-1 accuracy (%): 86.8 to 90.9 (+4.1)Improved
add contact map to mesh vertex input of the interaction recognizer (ablation)fIA interaction recognizer · FPHA testtop-1 accuracy (%): 96.8 to 98.4 (+1.6, 2000 samples only)No clear effect
use mesh vertices and contact maps instead of raw hand poses for interaction recognition (ablation)fIA interaction recognizer · H2O testtop-1 accuracy (%): 89.3 to 90.9 (+1.6)Improved
use mesh vertices and contact maps instead of raw hand poses for interaction recognition (ablation)fIA interaction recognizer · FPHA testtop-1 accuracy (%): 91.9 to 98.4 (+6.5) with full contact; pose-only is higher than mesh without contact (96.8 vs 91.9 comparison shows gainMixed
reduce contact map neighbor search to 10 nearest vertices instead of all-pairs distance (ablation)contact map generation · H2O testtop-1 accuracy (%): 90.9 to 82.2 (-8.7)Hurt
replace Transformer pose estimator with CNN pose estimator (Kwon et al.) in interaction pipeline (ablation)pose estimation and fIA pipeline · H2O testtop-1 accuracy (%): 90.9 to 80.9 (-10.0)Hurt
replace Transformer interaction recognizer with LSTM (ablation)fIA interaction recognizer · H2O testtop-1 accuracy (%): 90.9 to 82.6 (-8.3)Hurt
Transformer-based unified framework (H2OTR) vs prior methods on pose estimationH2OTR · H2O and FPHA testmean end-point error (mm): H2O right hand 25.8 vs 37.2 (Kwon et al.); FPHA object 21.0 vs 24.9 (Tekin et al.)Improved
Transformer-based interaction recognition with contact maps vs prior interaction recognizersH2OTR · H2O testtop-1 accuracy (%): 90.9 vs 86.4 (Wen et al.)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy