Research map / Architectures and efficiency / Attention and transformers
TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Models
Pooyan Rahmanzadehgervi, Hung Huy Nguyen, Rosanne Liu, Long Mai, Anh Totti Nguyen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention supervision (ablation) | TAB4IDC (ViT-B/16) · OpenImages-I captioning | CIDEr: 324.4 -> 335.6 | Improved |
| add attention supervision (ablation) | TAB (ViT-B/16) · OpenImages-I localization (PG+) | PG+ mean accuracy: 61.73 -> 99.19 | Improved |
| add attention bottleneck (TAB) in place of CLIP4IDC MHSA (ablation) | CLIP4IDC (ViT-B/16) · OpenImages-I localization (PG+) | PG+ mean accuracy: 52.98 -> 99.19 | Improved |
| replace MHSA with TAB | CLIP4IDC (ViT-B/16) · CLEVR-Change localization (PG+) | PG+ mean accuracy: 84.47 -> 96.75 | Improved |
| replace MHSA with TAB (ViT-B/32) | CLIP4IDC · CLEVR-Change captioning | CIDEr: 150.7 -> 149.6 | No clear effect |
| replace MHSA with TAB (ViT-B/16) | CLIP4IDC · CLEVR-Change captioning | CIDEr: 144.7 -> 153.0 | Improved |
| replace MHSA with TAB (ViT-B/32) | CLIP4IDC · STD captioning | BERTScore: 29.4 -> 22.6 | Hurt |
| replace MHSA with TAB (ViT-B/16) | CLIP4IDC · STD captioning | BERTScore: 23.0 -> 28.3 | Improved |
| use higher-resolution ViT-B/16 instead of B/32 (ablation) | TAB · OpenImages-I localization (PG+) | PG+ mean accuracy: 98.11 -> 99.19 | Improved |
| replace attention map with groundtruth (CorrectAttention) (ablation) | TAB4IDC · OpenImages-I editing (attention supervised, ViT-B/16) | object name accuracy: 88.92 -> 91.49 | Improved |
| replace attention map with groundtruth (CorrectAttention) (ablation) | TAB4IDC (CLEVR-Change) · CLEVR-Change editing | change accuracy: 94.30 -> 100.0 | Improved |
| replace attention map with zeros (ZeroAttention) (ablation) | TAB4IDC · OpenImages-I editing (attention supervised, ViT-B/16) | change accuracy: 99.93 -> 0.0 | Hurt |
Closest papers by meaning
- Head Pursuit: Probing Attention Specialization in Multimodal Transformers (NeurIPS 2025 · similarity 0.90)
- No Head Left Behind – Multi-Head Alignment Distillation for Transformers (AAAI 2024 · similarity 0.89)
- See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.89)
- A-VL: Adaptive Attention for Large Vision-Language Models (AAAI 2025 · similarity 0.89)
- Attention Alignment Between Humans and Vision-Language Models (NeurIPS 2026 · similarity 0.88)
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
