Research map / Vision-language and multimodal / Visual question answering
TRAR: Routing the Attention Spans in Transformer for Visual Question Answering
Yiyi Zhou, Tianhe Ren, Chaoyang Zhu, Xiaoshuai Sun, Jianzhuang Liu, Xinghao Ding, Mingliang Xu, Rongrong Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TRAR attention-span routing (soft) to the multi-modal Transformer | MCAN · VQA2.0 val | accuracy (All): 67.3 -> 67.7 | Improved |
| add TRAR attention-span routing (soft) to the one-stage REC Transformer | DarkNet + Transformer · RefCOCO val | IoU accuracy: 75.8 -> 77.6 | Improved |
| use mask-based routing (TRAR) instead of conventional multi-module routing | MCAN · VQA2.0 val | Transformer parameters: 67M -> 45M | Improved |
| use conventional (separate-module) routing instead of the plain Transformer | DarkNet + Transformer · RefCOCO val | IoU accuracy: 75.8 -> 75.1 | Hurt |
| add a 1st-order local attention mask to the routing space (ablation) | TRAR-S · VQA2.0 val | accuracy (All): 67.3 -> 67.6 | Improved |
| add a 1st-order local attention mask to the routing space (ablation) | TRAR-S · RefCOCO val | IoU accuracy: 75.8 -> 76.2 | Improved |
| add a 3rd-order mask (routing space [0,1,2] -> [0,1,2,3]) (ablation) | TRAR-S · VQA2.0 val | accuracy (All): 67.7 -> 67.6 | No clear effect |
| add a 3rd-order mask (routing space [0,1,2] -> [0,1,2,3]) (ablation) | TRAR-S · RefCOCO val | IoU accuracy: 76.7 -> 77.6 | Improved |
| add a 3rd-order mask (routing space [0,1,2] -> [0,1,2,3]) (ablation) | TRAR-H · RefCOCO val | IoU accuracy: 76.9 -> 76.1 | Hurt |
| add a 4th-order mask (routing space [0,1,2,3] -> [0,1,2,3,4]) (ablation) | TRAR-S · RefCOCO val | IoU accuracy: 77.6 -> 77.5 | No clear effect |
| use 16x16 grid features instead of the 8x8 resolution | TRAR-S · VQA2.0 test-dev | accuracy (All): 72.00 -> 72.62 | Improved |
| add TRAR (soft) to the Transformer on CLEVR | MCAN · CLEVR | overall accuracy: 98.54 -> 99.00 | Improved |
Closest papers by meaning
- Question Aware Vision Transformer for Multimodal Reasoning (CVPR 2024 · similarity 0.87)
- AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers (ICML 2024 · similarity 0.86)
- Rethinking Token Reduction for Large Vision-Language Models (CVPR 2026 · similarity 0.86)
- AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition (CVPR 2026 · similarity 0.86)
- Cross-modal Information Flow in Multimodal Large Language Models (CVPR 2025 · similarity 0.86)
- Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
