Research map / Vision-language and multimodal / Human and embodied modeling
On Guiding Visual Attention With Language Specification
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add language-grounded spatial attention supervision | ResNet50 classifier · Waterbirds-95% | worst-group accuracy: 73.66 ± 2.00 -> 76.54 ± 2.40 | Helped |
| Add language-grounded spatial attention supervision | ResNet50 classifier · Waterbirds-100% | worst-group accuracy: 41.56 ± 6.76 -> 56.71 ± 3.92 | Helped |
| Add language-grounded spatial attention supervision | ResNet50 classifier · Red Meat (Food-101) | test accuracy: 69.44 ± 1.12 -> 71.20 ± 0.84 | Helped |
| Add language-grounded spatial attention supervision | ResNet50 classifier · MSCOCO-ApparentGender | Outcome Divergence: 0.071 -> 0.022 | Helped |
| Add language-grounded spatial attention supervision | ResNet50 classifier · Waterbirds-95% Pointing Game | Pointing Game accuracy: 59.98 -> 69.38 | Helped |
| Add language-grounded spatial attention supervision | ResNet50 classifier · Waterbirds-100% Pointing Game | Pointing Game accuracy: 46.48 -> 59.27 | Helped |
| Use language specification for the background-classification task instead of bird-classification guidance | ResNet50 classifier · Waterbirds-100% Backgrounds | per-group accuracy: +2.22% | Helped |
| Use language specification for the background-classification task instead of bird-classification guidance | ResNet50 classifier · Waterbirds-100% Backgrounds | worst-group accuracy: +7.32% | Helped |
| Use CLIP ViT rather than CLIP ResNet50 to generate language attention (ablation) | RRR classifier attention on Waterbirds-95% · Waterbirds-95% validation | per-class accuracy: 87.26 -> 88.25 | Helped |
| Use ABN feed-forward attention rather than RRR classifier attention (ablation) | Language-guided classifier on Waterbirds-95% · Waterbirds-95% validation | per-class accuracy: 87.26 -> 86.10 | Hurt |
| Supervise classifier GradCAM directly rather than using RRR (ablation) | Language-guided classifier on Waterbirds-95% · Waterbirds-95% validation | per-class accuracy: 87.26 -> 84.44 | Hurt |
| Use CLIP ViT rather than CLIP ResNet50 to generate language attention (ablation) | RRR classifier attention on MSCOCO-ApparentGender · MSCOCO-ApparentGender validation | Ratio Delta: 0.235 -> 0.341 | Hurt |
Closest papers by meaning
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.90)
- What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.88)
- Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.88)
- Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.88)
- Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.88)
- See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.