Research map / Vision-language and multimodal / Visual question answering
Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following
Qiaomu Miao, Alexandros Graikos, Jingwei Zhang, Sounak Mondal, Minh Hoai, Dimitris Samaras
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| refine Grad-CAM heatmaps with a diffusion model instead of using them raw as pseudo labels (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.237 -> 0.179 | Improved |
| use Grad-CAM VQA prior as the starting point of diffusion refinement (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.191 -> 0.179 | Improved |
| use diffusion refinement instead of a Gaussian placed at the Grad-CAM argmax (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.207 -> 0.179 | Improved |
| use diffusion model instead of a direct U-Net mapping from Grad-CAM to ground truth (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.190 -> 0.179 | Improved |
| start refinement from intermediate noise step 250 rather than 200 or 300 (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.184 (t=200), 0.186 (t=300) -> 0.179 (t=250) | Improved |
| use 5 inference steps instead of 2 in diffusion refinement (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.179 -> 0.180 | No clear effect |
| train the diffusion model with Mean Teacher on unlabeled data (GCDR-MT) (ablation) | GCDR · GazeFollow (10% labels) | Avg. Dist.: 0.179 -> 0.172 | Improved |
| feed Grad-CAM heatmaps as direct input to the gaze model (ablation) | VAT · GazeFollow (10% labels) | Avg. Dist.: 0.202 -> 0.188 | Improved |
| refine Grad-CAM pseudo labels with diffusion instead of VAT teacher pseudo labels (GCDR vs Semi-VAT) | VAT student · GazeFollow (5% labels) | Avg. Dist.: 0.222 -> 0.201 | Improved |
| increase amount of unlabeled data from 10% to 90% of GazeFollow (ablation) | GCDR-MT · GazeFollow (10% labels) | Avg. Dist.: 0.183 -> 0.172 | Improved |
| apply Mean Teacher directly to the Miao et al. model | Miao et al. model · GazeFollow (10% labels) | Avg. Dist.: 0.190 -> 0.217 | Hurt |
| adapt GCDR to the Lian et al. base model with 10% labels | GCDR-MT · GazeFollow (10% labels, Lian base model) | Avg. Dist.: 0.216 -> 0.171 | Improved |
Closest papers by meaning
- Toward Semantic Gaze Target Detection (NeurIPS 2024 · similarity 0.88)
- Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models (NeurIPS 2026 · similarity 0.87)
- ST-DiffEye: Diffusion-based Continuous Gaze Generation via Joint Scanpath-Trajectory Modeling (NeurIPS 2026 · similarity 0.86)
- In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting (NeurIPS 2025 · similarity 0.85)
- DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images (NeurIPS 2025 · similarity 0.85)
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
