Research map / Label-efficient and robust learning / Self-supervised and contrastive
Bootstrap AutoEncoders With Contrastive Paradigm for Self-supervised Gaze Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BeCa, a self-supervised full-face gaze estimator that adds a batch-level InfoNCE contrastive regularizer to autoencoder reconstruction, plus an InfoMSE loss that decouples representation quality from pixel reconstruction.
Problem
Self-supervised gaze estimation has mostly been restricted to eye images; contrastive methods are ineffective for full-face gaze data and generative autoencoders on full-face images collapse to trivial, gaze-irrelevant solutions.
Approach
Decompose the autoencoder likelihood to expose a mutual-information term, add an InfoNCE contrastive loss on latent features to regularize semantics, and replace the vanilla MSE with InfoMSE, a contrastive-style label-matching loss over batch ground-truth labels, so representation learning is not tied to reconstruction quality.
Key results
- BeCa achieves 5.76 gaze error on MPII with ResNet-18 under 100-shot linear probing, versus 6.60 for AutoEncoder (ERC) and 8.7 to 12.1 for other baselines, and BeCa is about 20% lower than CE (7.2) on MPII.
- BeCa-InfoMSE reaches 5.44 on MPII and 6.12 on Columbia with ResNet-18, improving on BeCa by nearly 5% across datasets.
- On Gaze360 with ViT-tiny, BeCa reduces 100-shot linear probing error to 16.23 from 24.25 for AutoEncoder (ERC), over 30% improvement.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add contrastive regularization to the autoencoder (ablation) | BeCa · MPII | gaze error: 6.60 -> 5.76 | Helped |
| Add contrastive regularization to the autoencoder (ablation) | BeCa · Gaze360 | gaze error: 26.87 -> 22.54 | Helped |
| Replace MSE with InfoMSE (ablation) | BeCa · MPII | gaze error: 5.76 -> 5.44 | Helped |
| Replace MSE with InfoMSE (ablation) | BeCa · Gaze360 | gaze error: 22.54 -> 21.75 | Helped |
| Use InfoMSE without contrastive learning (ablation) | AutoEncoder (ERC) · MPII | gaze error: 6.60 -> 5.95 | Helped |
| Use InfoMSE without contrastive learning (ablation) | AutoEncoder (ERC) · Gaze360 | gaze error: 26.87 -> 24.93 | Helped |
| Reconstruct full-face images instead of eye regions (ablation) | BeCa · MPII | gaze error: 5.76 -> 8.62 | Hurt |
| Reconstruct full-face images instead of eye regions (ablation) | BeCa-InfoMSE · MPII | gaze error: 5.44 -> 8.51 | Hurt |
| Increase linear-probing samples from 50 to 200 (ablation) | BeCa-InfoMSE · Columbia | gaze error: 6.75 -> 6.05 | Helped |
| Increase linear-probing samples from 50 to 200 (ablation) | BeCa-InfoMSE · MPII | gaze error: 6.15 -> 5.23 | Helped |
| Vary the BeCa loss weight γ (ablation) | BeCa · Xgaze | gaze error: γ = 5: 14.73; γ = 3: 14.13; γ = 1: 14.59; γ = 0.2: 14.63 | Mixed |
| Vary the InfoMSE loss weight β (ablation) | BeCa-InfoMSE · Gaze360 | gaze error: β = 0.01: 20.27; β = 0.05: 20.44; β = 0.1: 21.27; β = 1: 21.77 | Mixed |
Surprising findings
- Reconstruction quality is weakly tied to representation quality: BeCa-InfoMSE reached 4.72 gaze error with 0.24 MSE, while BeCa reached 5.58 error with 0.029 MSE.
- Full-face reconstruction (FRC) performs worse than eye-region reconstruction (ERC) for the vanilla autoencoder, since facial detail suppresses gaze information.
Limitations
- Gains on Columbia depend on initializing from an ETH-Xgaze pretrained model, and cross-dataset results rely on Xgaze source data.
- Hyperparameters gamma and beta were tuned on test-set-style ablations, and results are reported as 100-shot linear probing averages rather than standard protocols.
- Relies on a single augmentation family (color jitter and grayscale), which the paper notes is limited for spatial transforms.
Experimental setup
Datasets: ColumbiaGaze MPIIFaceGaze Gaze360 ETH-Xgaze EVE
Benchmarks: ColumbiaGaze MPIIFaceGaze Gaze360 ETH-Xgaze
Metrics: gaze angular error (degrees)
Baselines: ImageNet-Pretrained AutoEncoder (FRC) AutoEncoder (ERC) SimCLR Barlow Twins Masked AutoEncoder CE GazeCLR
Builds on: SimCLR Barlow Twins Masked AutoEncoder CE (Sun et al., 2021) InfoNCE
Closest papers by meaning
- Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · similarity 0.88)
- OmniGaze: Reward-inspired Generalizable Gaze Estimation in the Wild (NeurIPS 2025 · similarity 0.86)
- Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process (AAAI 2024 · similarity 0.85)
- Test-Time Personalization with Meta Prompt for Gaze Estimation (AAAI 2024 · similarity 0.85)
- Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders (CVPR 2025 · similarity 0.85)
- Unsupervised Gaze Representation Learning from Multi-view Face Images (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.