Research map / Video / Video-language understanding
Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning
Lifeng Fan, Wenguan Wang, Siyuan Huang, Xinyu Tang, Song-Chun Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a second message passing iteration (2 instead of 1) (ablation) | spatial reasoning graph · VACATION atomic-level | top-1 acc: 33.67 -> 55.02 | Improved |
| use 2 iterations instead of 3 (ablation) | spatial reasoning graph · VACATION atomic-level | top-1 acc: 44.52 -> 55.02 | Improved |
| use 2 iterations instead of 4 (ablation) | spatial reasoning graph · VACATION atomic-level | top-1 acc: 48.35 -> 55.02 | Improved |
| add LSTM temporal reasoning instead of Conv3d layers (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 53.73 -> 55.02 | Improved |
| add LSTM temporal reasoning instead of Conv3d layers (ablation) | Ours-full · VACATION atomic-level | top-2 acc: 80.33 -> 76.45 | Hurt |
| learn adjacency matrix explicitly with attention supervision instead of implicitly (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 33.74 -> 55.02 | Improved |
| use ResNet50 4096-d node features instead of VGG16 (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 49.91 -> 55.02 | Improved |
| use ResNet50 features instead of PRNet face keypoints (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 39.59 -> 55.02 | Improved |
| compress ResNet50 features to 192-d (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 55.02 -> 53.72 | No clear effect |
| use GNN with spatial structure instead of adjacency-matrix-only classifier (ablation) | AdjMat-only · VACATION atomic-level | top-1 acc: 32.64 -> 55.02 | Improved |
| add second adjacency-matrix branch beside the GNN branch (ablation) | Ours-full · VACATION atomic-level | top-1 acc: 55.02 -> 49.57 | Hurt |
| use graph model instead of CNN+LSTM baseline | CNN+LSTM · VACATION atomic-level | top-1 acc: 24.65 -> 55.02 | Improved |
Closest papers by meaning
- MTGS: A Novel Framework for Multi-Person Temporal Gaze Following and Social Gaze Prediction (NeurIPS 2024 · similarity 0.89)
- Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models (NeurIPS 2026 · similarity 0.86)
- Sharingan: A Transformer Architecture for Multi-Person Gaze Following (CVPR 2024 · similarity 0.86)
- OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following (NeurIPS 2026 · similarity 0.85)
- EyeGraph: Modularity-aware Spatio Temporal Graph Clustering for Continuous Event-based Eye Tracking (NeurIPS 2024 · similarity 0.85)
- StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
