Research map / 3D vision / 3D generation
GazeInterpreter: Parsing Eye Gaze to Generate Eye-Body-Coordinated Narrations
Qing Chang, Zhiming Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GazeInterpreter narrations instead of Nymeria atomic narrations as MotionGPT input | MotionGPT · Nymeria text-driven motion generation (all scenes) | FID: 8.131 -> 7.468 | Improved |
| use GazeInterpreter narrations on low-level scenes | MotionGPT · Nymeria text-driven motion generation (low-level scenes) | FID: 7.458 -> 6.801 | Improved |
| use GazeInterpreter narrations as zero-shot action anticipation input | GazeInterpreter · Nymeria action anticipation (all scenes) | cosine similarity: 0.459 -> 0.506 | Improved |
| use GazeInterpreter narrations for behavior summarization on high-level scenes | GazeInterpreter · Nymeria behavior summarization (high-level scenes) | cosine similarity: 0.395 -> 0.490 | Improved |
| add hierarchical structure to integrate gaze semantics with body motion (ablation) | GazeInterpreter · Nymeria text-driven motion generation (all scenes) | FID: 8.135 -> 6.634 MM Dist; 9.124 -> 7.468 FID | Improved |
| add symbolic gaze parser instead of raw gaze signal (ablation) | GazeInterpreter · Nymeria text-driven motion generation (all scenes) | FID: 7.893 -> 7.468 | Improved |
| add self-correcting loop (ablation) | GazeInterpreter · Nymeria text-driven motion generation (all scenes) | FID: 7.831 -> 7.468 | Improved |
| add each evaluation dimension cumulatively in self-correcting loop (ablation) | GazeInterpreter · Nymeria text-driven motion generation (all scenes) | FID: 7.831 -> 7.468 | Improved |
| increase observation window size W beyond 2 (ablation) | GazeInterpreter · Nymeria text-driven motion generation (all scenes) | quality vs inference cost | No clear effect |
Closest papers by meaning
- EyEar: Learning Audio Synchronized Human Gaze Trajectory Based on Physics-Informed Dynamics (AAAI 2025 · similarity 0.89)
- ST-DiffEye: Diffusion-based Continuous Gaze Generation via Joint Scanpath-Trajectory Modeling (NeurIPS 2026 · similarity 0.89)
- OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following (NeurIPS 2026 · similarity 0.88)
- Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models (NeurIPS 2026 · similarity 0.88)
- StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos (NeurIPS 2026 · similarity 0.87)
- Gaze-VLM: Bridging Gaze and VLMs through Attention Regularization for Egocentric Understanding (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
