Research map / 3D vision / 3D generation
3D Prior Is All You Need: Cross-Task Few-shot 2D Gaze Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts a pre-trained 3D gaze estimator (GazeTR) to 2D screen gaze on unseen devices from about 10 images, using a learnable physics-based projection module and dynamic pseudo-labelling for flipped images.
Problem
Few-shot 2D gaze estimation on unseen devices without screen calibration or source data, bridging the domain gap between 3D gaze predictions and 2D screen gaze labels.
Approach
Model the unknown screen pose as six learnable parameters (rotation and translation) in a differentiable ray-plane projection on top of a pre-trained 3D gaze network, and generate flip pseudo-labels dynamically by inverting the projection into 3D, flipping there, and aligning the drifting camera frame with an SVD-learned transformation.
Key results
- With 10 training samples, the method reaches 43.4 mm on EVE, 56.7 mm on MPIIGaze and 35.7 mm on GazeCapture, beating EFE (64.9, 100.2, 48.5) and IVGaze (177.7, 132.2, 68.1).
- Removing the transformation T in the pseudo-labelling strategy raises error on MPIIGaze and EVE substantially, showing the alignment step is key.
- Error decreases steadily as the number of training images grows from 3 to 50, and each training run takes about 1.2 minutes on an RTX 3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full method instead of EFE with 10 samples | Cross-task few-shot 2D gaze estimation · EVE | Euclidean distance (mm): 64.9 -> 43.4 | Helped |
| Use the full method instead of EFE with 10 samples | Cross-task few-shot 2D gaze estimation · MPIIGaze | Euclidean distance (mm): 100.2 -> 56.7 | Helped |
| Use the full method instead of EFE with 10 samples | Cross-task few-shot 2D gaze estimation · GazeCapture | Euclidean distance (mm): 48.5 -> 35.7 | Helped |
| Add dynamic pseudo-labeling to the projection-only method (ablation) | Cross-task few-shot 2D gaze estimation · EVE | Euclidean distance (mm): 46.6 -> 45.3 | Helped |
| Add dynamic pseudo-labeling to the projection-only method (ablation) | Cross-task few-shot 2D gaze estimation · MPIIGaze | Euclidean distance (mm): 60.3 -> 57.9 | Helped |
| Add dynamic pseudo-labeling to the projection-only method (ablation) | Cross-task few-shot 2D gaze estimation · GazeCapture | Euclidean distance (mm): 36.8 -> 35.7 | Helped |
| Add uncertainty minimization across jittered images (ablation) | Projection with dynamic pseudo-labeling · EVE | Euclidean distance (mm): 45.3 -> 43.4 | Helped |
| Add uncertainty minimization across jittered images (ablation) | Projection with dynamic pseudo-labeling · MPIIGaze | Euclidean distance (mm): 57.9 -> 56.7 | Helped |
| Add uncertainty minimization across jittered images (ablation) | Projection with dynamic pseudo-labeling · GazeCapture | Euclidean distance (mm): 35.7 -> 35.7 | No clear effect |
| Add the coordinate-alignment transformation T to dynamic pseudo-labeling (ablation) | Cross-task few-shot 2D gaze estimation · MPIIGaze | Euclidean distance (mm): 100 -> 56.7 | Helped |
| Add the coordinate-alignment transformation T to dynamic pseudo-labeling (ablation) | Cross-task few-shot 2D gaze estimation · EVE | Euclidean distance (mm): 61 -> 43.4 | Helped |
| Add the coordinate-alignment transformation T to dynamic pseudo-labeling (ablation) | Cross-task few-shot 2D gaze estimation · GazeCapture | Euclidean distance (mm): 34.3 -> 35.7 | Hurt |
Surprising findings
- Omitting the coordinate transformation T improved results on GazeCapture, which the authors attribute to the initial screen pose coinciding with the true pose there.
- Direct fine-tuning of the 3D network with an MLP mapping to 2D fails to outperform direct projection, showing the 3D-to-2D domain gap is hard to bridge without the projection structure.
Limitations
- Fails when input images lack visible faces, as the 3D gaze can become erratic and the projection yields extreme values, especially when gaze is near-parallel to the screen.
- Requires collecting a few calibration-like samples from the target device before adaptation.
- Pseudo-label alignment relies on predictions from the 3D network as anchors, which may introduce bias.
Experimental setup
Datasets: Gaze360 MPIIGaze EVE GazeCapture
Benchmarks: MPIIGaze EVE GazeCapture
Metrics: Euclidean distance (mm)
Baselines: EFE IVGaze Direct Projection Direct Learning RAT
Builds on: GazeTR EFE RAT Image normalization for gaze estimation
Closest papers by meaning
- Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · similarity 0.88)
- GA3CE: Unconstrained 3D Gaze Estimation with Gaze-Aware 3D Context Encoding (CVPR 2025 · similarity 0.88)
- De^2Gaze: Deformable and Decoupled Representation Learning for 3D Gaze Estimation (CVPR 2025 · similarity 0.87)
- From Feature to Gaze: A Generalizable Replacement of Linear Layer for Gaze Estimation (CVPR 2024 · similarity 0.85)
- UVAGaze: Unsupervised 1-to-2 Views Adaptation for Gaze Estimation (AAAI 2024 · similarity 0.85)
- OmniGaze: Reward-inspired Generalizable Gaze Estimation in the Wild (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.