Research map / Video / Action recognition
In the Eye of the Beholder: Gaze and Actions in First Person Video
Yin Li, Miao Liu, James Matthew Rehg
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add probabilistic (sampled) gaze modeling instead of deterministic Gaze MLE (ablation) | gaze-aware I3D · EGTEA Gaze+ all splits | action accuracy and gaze F1: 1.0% for action recognition and 6.9% for gaze estimation | Improved |
| add human gaze supervision (vs uniform prior) (ablation) | probabilistic gaze model · EGTEA Gaze+ Split1 | gaze F1: 11.63 -> 34.01 | Improved |
| add human gaze supervision (vs uniform prior) (ablation) | probabilistic gaze model · EGTEA Gaze+ Split1 | action accuracy: 56.50 -> 57.20 | Improved |
| add probabilistic attention without gaze supervision to I3D (ablation) | I3D joint · EGTEA Gaze+ | action accuracy: +0.4% | Improved |
| pool features directly with human gaze (ablation) | I3D joint · EGTEA Gaze+ Split1 | action accuracy: 55.76 -> 53.74 | Hurt |
| use center prior as attention map (ablation) | I3D joint · EGTEA Gaze+ Split1 | action accuracy: 55.76 -> 52.48 | Hurt |
| add egocentric cue stream (EgoConv) (ablation) | I3D joint · EGTEA Gaze+ Split1 | action accuracy: 55.76 -> 54.19 | Hurt |
| use joint training of RGB and flow streams instead of RGB only (ablation) | I3D · EGTEA Gaze+ Split1 | action accuracy: 50.08 -> 55.76 | Improved |
| use flow stream instead of RGB stream (ablation) | I3D · EGTEA Gaze+ Split1 | action accuracy: 50.08 -> 45.78 | Hurt |
| use KL loss instead of cross entropy for gaze (ablation) | gaze estimation · EGTEA Gaze+ Split1 | gaze F1: 31.33 -> 34.46 | Improved |
| add dropout 0.7 and longer training (ablation) | I3D joint and our model · EGTEA Gaze+ Split1 | action accuracy: 49.79 -> 53.55 | Improved |
| add dropout and longer training (ablation) | our model · EGTEA Gaze+ Split1 | gaze F1: 32.66 -> 36.01 | Improved |
Closest papers by meaning
- In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video (ECCV 2018 · similarity 0.96)
- Following Gaze in Video (ICCV 2017 · similarity 0.87)
- Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders (CVPR 2025 · similarity 0.87)
- Sharingan: A Transformer Architecture for Multi-Person Gaze Following (CVPR 2024 · similarity 0.86)
- Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · similarity 0.85)
- GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
