Research map / 3D vision / 3D generation
GazeGene: Large-scale Synthetic Gaze Dataset with 3D Eyeball Annotations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GazeGene, a large-scale synthetic full-face gaze dataset (1M+ images from 56 MetaHuman subjects, 9 cameras) with accurate 3D eyeball annotations (eyeball center/radius, pupil, iris, optical and visual axes), and validates it on gaze estimation and eyeball-structure tasks.
Problem
Existing large-scale appearance-based gaze datasets have annotation errors from calibration and head pose estimation, provide only a single gaze vector, and lack 3D eyeball structure annotations (eyeball center, radius, pupil, optical/visual axes), limiting research on deep-learning gaze methods.
Approach
Render photo-realistic full-face images of MetaHuman virtual characters in Unreal Engine with randomized gaze, head pose, expression, blink, pupil size and lighting, and compute exact 3D eyeball annotations from the source meshes (sphere fitting, kappa-angle-based optical/visual axis conversion), which real datasets cannot measure.
Key results
- Within-domain: ResNet-50 achieves 3.36° on GazeGene vs 5.32° on ETH-XGaze, and GazeGene gives the best error across all five models tested.
- Cross-domain: a ResNet-18 trained on GazeGene reaches 12.87° on ETH-XGaze, versus 15.42° for the reverse direction result shown for training on ETH-XGaze, per Table 3 (the table reports test errors per training set).
- Optical axis estimation for ResNet-50 has 4.19° error; visual axis estimation is about 0.1° worse, and applying GT kappa to the optical axis gives slightly better gaze accuracy than direct visual axis regression.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train ResNet gaze estimators on GazeGene rather than real-world datasets | ResNet-18 and ResNet-50 · ETH-XGaze cross-domain evaluation | gaze estimation error (degrees): 12.87; 12.37 | Helped |
| Regress visual axis instead of optical axis and applying ground-truth kappa | ResNet-18 and ResNet-50 · GazeGene test set | angular error (degrees): 4.93 .30 vs 4.89 .20; 4.22 .21 vs 4.20 .29 | Hurt |
| Apply ground-truth kappa to optical-axis predictions | ResNet-18 and ResNet-50 · GazeGene test set | angular error (degrees): 4.93 .30 -> 4.89 .20; 4.22 .21 -> 4.20 .29 | Helped |
| Estimate gaze by reconstructing 3D eyeball structures | ResNet-18 and ResNet-50 on GazeGene · GazeGene eyeball structure estimation | optical axis error (degrees): 4.98; 4.72 | Helped |
| Estimate gaze vergence depth from remote face images | ResNet-50 on GazeGene · GazeGene samples with vergence depth within [0.2m, 1.5m] | average depth error (cm): 27cm | Mixed |
Surprising findings
- Visual axis estimation is consistently harder than optical axis estimation, even for data-driven models, which suggests kappa angle variation across subjects limits learnability.
Limitations
- Dataset has no random backgrounds, limiting variety.
- Synthetic-to-real domain gap remains; ResNet models are about 2 degrees worse on GazeGene than on ETH-XGaze in within-domain tests.
- Experiments use only basic models and straightforward designs, so 3D eyeball task results are a lower bound; Epic license forbids use as generative AI training input.
Experimental setup
Datasets: GazeGene ETH-XGaze MPIIFaceGaze Gaze360 EyeDiap EVE
Benchmarks: GazeGene ETH-XGaze MPIIFaceGaze Gaze360 EyeDiap EVE
Metrics: angular error (degrees) 2D pixel error 3D error (cm) depth error (cm)
Baselines: FullFace Dilated-Net GazeTR 3DGazeNet ETH-XGaze MPIIFaceGaze Gaze360 EyeDiap EVE
Builds on: MetaHuman Unreal Engine ETH-XGaze UnityEyes
Closest papers by meaning
- De^2Gaze: Deformable and Decoupled Representation Learning for 3D Gaze Estimation (CVPR 2025 · similarity 0.85)
- From Feature to Gaze: A Generalizable Replacement of Linear Layer for Gaze Estimation (CVPR 2024 · similarity 0.85)
- Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · similarity 0.85)
- Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders (CVPR 2025 · similarity 0.85)
- OmniGaze: Reward-inspired Generalizable Gaze Estimation in the Wild (NeurIPS 2025 · similarity 0.85)
- Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.