Research map / Large language models / Reasoning and chain of thought
AiR: Attention with Reasoning Capability
Shi Chen, Ming Jiang, Jinhui Yang, Qi Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AiR-M progressive attention supervision | UpDown · GQA test-dev | accuracy (%): 51.31 -> 53.46 | Improved |
| add AiR-M progressive attention supervision | MUTAN · GQA test-dev | accuracy (%): 50.78 -> 51.81 | Improved |
| add AiR-M progressive attention supervision | BAN · GQA test-dev | accuracy (%): 50.14 -> 53.36 | Improved |
| add PAAN adversarial attention supervision | UpDown · GQA test-dev | accuracy (%): 51.31 -> 48.03 | Hurt |
| add HAN human-like attention supervision | UpDown · GQA test-dev | accuracy (%): 51.31 -> 49.96 | Hurt |
| add ASM supervision from objects mined from language | UpDown · GQA test-dev | accuracy (%): 51.31 -> 52.96 | Improved |
| use progressive multi-step supervision instead of single-glimpse joint supervision (ablation) | AiR-M on UpDown · GQA test-dev | accuracy (%): 52.84 -> 53.46 | Improved |
| add attention supervision L_alpha to progressive reasoning-operation supervision (ablation) | AiR-M on UpDown · GQA test-dev | accuracy (%): 50.01 -> 53.46 | Improved |
| add reasoning-operation supervision L_r to progressive attention supervision (ablation) | AiR-M on UpDown · GQA test-dev | accuracy (%): 50.33 -> 53.46 | Improved |
| increase operation loss weight phi from 0.5 to 10 (ablation) | AiR-M on UpDown · GQA balanced val | accuracy (%): 62.57 -> 61.39 | Hurt |
| vary phi from 0.01 to 1 (ablation) | AiR-M on UpDown · GQA balanced val | accuracy (%): 62.02 / 62.11 / 62.57 / 61.93 | No clear effect |
| use object-based attention instead of spatial attention (ablation) | UpDown VQA attention · AiR-E on GQA subset | AiR-E (query): 0.725 -> 3.241 | Improved |
Closest papers by meaning
- Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning (NeurIPS 2025 · similarity 0.85)
- Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas (ICML 2025 · similarity 0.85)
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.85)
- Caption This, Reason That: VLMs Caught in the Middle (NeurIPS 2025 · similarity 0.85)
- Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning Models (ICML 2026 · similarity 0.84)
- MoH: Multi-Head Attention as Mixture-of-Head Attention (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
