Research map / Video / Video segmentation and tracking
Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video
Miao Liu, Siyu Tang, Yin Li, James M. Rehg
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add stochastic units to the joint model (ablation) | joint motor attention and hotspots model · EGTEA action anticipation (action) | top-1 accuracy (%): 35.69 -> 36.60 | Improved |
| add stochastic units to the joint model (ablation) | joint motor attention and hotspots model · EPIC-Kitchens action anticipation, val split (action) | top-1 accuracy (%): 9.76 -> 10.38 | Improved |
| add stochastic units to the joint model (ablation) | joint model · Interaction hotspots EGTEA | F1: 23.26 -> 25.69 | Improved |
| add motor attention module to hotspots-only model (ablation) | I3D-Res50 · EGTEA action anticipation (action) | top-1 accuracy (%): 35.50 -> 36.60 | Improved |
| add interaction hotspots module to motor-only model (ablation) | I3D-Res50 · EGTEA action anticipation (action) | top-1 accuracy (%): 36.49 -> 36.60 | Improved |
| add interaction hotspots module to motor-only model (ablation) | I3D-Res50 · EPIC-Kitchens action anticipation, val split (action) | top-1 accuracy (%): 10.21 -> 10.38 | Improved |
| add joint motor attention and hotspots modeling to I3D-Res50 | I3D-Res50 · EGTEA action anticipation (action) | top-1 accuracy (%): 34.82 -> 36.60 | Improved |
| add joint motor attention and hotspots modeling to I3D-Res50 | I3D-Res50 · EPIC-Kitchens action anticipation, val split (action) | top-1 accuracy (%): 9.60 -> 10.38 | Improved |
| add deterministic joint detection without stochastic units (ablation) | I3D-Res50 · EPIC-Kitchens action anticipation, val split (action) | top-1 accuracy (%): 9.60 -> 9.76 | No clear effect |
| add joint modeling to I3D heatmap baseline (ablation) | I3D-Res50 · Interaction hotspots EGTEA | F1: 19.11 -> 25.69 | Improved |
| add fusion of RULSTM object stream (Ours+Obj) | Ours · EPIC-Kitchens anticipation seen s1 (action) | top-1 accuracy (%): 14.04 -> 15.42 | Improved |
| replace motor attention and hotspots with gaze module (ablation) | Ours · EGTEA action anticipation (action) | top-1 accuracy (%): 36.60 -> 35.31 | Hurt |
Closest papers by meaning
- Forecasting Action Through Contact Representations From First Person Video (TPAMI 2021 · similarity 0.92)
- Joint Hand Motion and Interaction Hotspots Prediction From Egocentric Videos (CVPR 2022 · similarity 0.92)
- AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipation (ECCV 2024 · similarity 0.92)
- Integrating Affordances and Attention Models for Short-Term Object Interaction Anticipation (TPAMI 2026 · similarity 0.91)
- Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning (ICLR 2025 · similarity 0.88)
- Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
