Research map / Reinforcement learning / Visual and representation RL
Gazing at Rewards: Eye Movements as a Lens into Human and AI Decision-Making in Hybrid Visual Foraging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Human psychophysics on hybrid visual foraging with eye tracking, plus a transformer-based Visual Forager (VF) trained with PPO that mimics human reward-driven eye movements and foraging biases without human training data.
Problem
How target values and prevalence shape foraging decisions and eye movements when observers search for multiple instances of multiple target types, and how to model this with AI.
Approach
A transformer actor-critic with eccentricity-dependent foveated vision, value modulation of feature-similarity maps, and a click token, trained with PPO in procedurally generated hybrid foraging environments using a two-stage curriculum, with no human data for training.
Key results
- Human Norm. Scores were 87.4%, 84.1%, and 93.1% across the three conditions; VF achieved 72.6%, 67.1%, and 81.6%, outperforming all baselines.
- VF's mean saccade size was 4.06 degrees versus 4.05 degrees for humans, without training on human eye movements.
- VF outperforms FeatOnly across all five OOD conditions but still lags behind humans.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Visual Forager instead of the baseline models | Hybrid visual foraging · In-domain conditions | Norm. Score | Helped |
| Use Visual Forager instead of FeatOnly | Hybrid visual foraging · OOD conditions | Norm. Score | Helped |
| Remove eccentricity-dependent layers from the visual feature extractor (ablation) | Visual Forager (VF) · Three in-domain conditions | saccade size | Hurt |
| Add visual feature modulation alongside value modulation | Visual Forager (VF) · Hybrid visual foraging | Norm. Score | Helped |
Surprising findings
- Humans and VF both overpick the highest-valued targets and underpick the lowest-valued ones, despite no human data in training.
- VF is less sensitive to medium rewards than humans, showing a sign discrepancy for second-highest-valued items.
Limitations
- VF still lags behind humans in cumulative reward and OOD flexibility.
- Experiments use simplified 2D grid worlds with a fixed 16x16 layout and only 4 target types in training.
Experimental setup
Datasets: procedurally generated hybrid foraging environments 2,400-item object pool (Brady et al.)
Benchmarks: in-domain UnValEqPre and UnValUnPre OOD EqValUnPre, UTargets, UValues, UItemNum, USetSize
Metrics: Normalized Score Click Bias Ratio saccade size distribution fixation reward analysis ScanMatch Fixation Edit Distance
Baselines: Chance FeatOnly MaxVal AvgVal DQN UpperBound IVSN Gazeformer
Builds on: PPO IVSN Gazeformer eccentricity-dependent pooling DQN
Closest papers by meaning
- Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.84)
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.83)
- Visual Persuasion: What Influences Decisions of Vision-Language Models? (ICML 2026 · similarity 0.83)
- EyEar: Learning Audio Synchronized Human Gaze Trajectory Based on Physics-Informed Dynamics (AAAI 2025 · similarity 0.83)
- Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input (ICML 2024 · similarity 0.83)
- ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning (CVPR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.