Research map / Architectures and efficiency / Attention and transformers
Visual Attention Emerges from Recurrent Sparse Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace RVT* self-attention with VARS-SD | RVT · ImageNet-C | error rate: 60.4 -> 58.3 | Helped |
| Replace RVT* self-attention with VARS-SD | RVT · ImageNet-R | accuracy: 41.7 -> 42.5 | Helped |
| Replace RVT* self-attention with VARS-SD | RVT · ImageNet-SK | accuracy: 28.7 -> 30.5 | Helped |
| Replace RVT* self-attention with VARS-SD | RVT · PGD | accuracy: 11.1 -> 11.4 | Helped |
| Replace RVT* self-attention with VARS-SD | RVT · ImageNet-A | accuracy: 11.1 -> 13.4 | Helped |
| Replace the global operator with VARS-SD | RVT · PACS Art | accuracy: 81.25 -> 86.08 | Helped |
| Replace self-attention with VARS-SD | RVT · PASCAL VOC 2012 val | mIoU: 39.92 -> 44.15 | Helped |
| Replace self-attention with VARS-SD | RVT · MIT1003 | NSS: 0.502 -> 0.678 | Helped |
| Use a static dictionary rather than a static+dynamic dictionary (ablation) | VARS on RVT · MIT1003 | NSS: 0.678 -> 0.737 | Helped |
| Use a dynamic dictionary rather than a static+dynamic dictionary (ablation) | VARS on RVT · MIT1003 | NSS: 0.678 -> 0.632 | Hurt |
| Set recurrent updates to 1 rather than 3 to 5 (ablation) | VARS · ImageNet-C | accuracy | Hurt |
| Set recurrent updates to 7 rather than 3 to 5 (ablation) | VARS · ImageNet-C | accuracy | Hurt |
Closest papers by meaning
- Faster Video Diffusion with Trainable Sparse Attention (NeurIPS 2025 · similarity 0.88)
- See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.87)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.87)
- Transformers Learn Faster with Semantic Focus (NeurIPS 2025 · similarity 0.87)
- Attention Distillation: A Unified Approach to Visual Characteristics Transfer (CVPR 2025 · similarity 0.87)
- Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshape (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.