Research map / Architectures and efficiency / Attention and transformers
Sharpen Focus: Learning With Attention Separability and Consistency
Lezi Wang, Ziyan Wu, Srikrishna Karanam, Kuan-Chuan Peng, Rajat Vikram Singh, Bo Liu, Dimitris N. Metaxas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inner-layer attention separation loss L_AS^in (ablation) | ResNet-50 · CUB-200-2011 | top-1 accuracy: 81.70 -> 85.15 | Improved |
| add attention consistency loss L_AC (ablation) | ResNet-50 + L_AS^in · CUB-200-2011 | top-1 accuracy: 85.15 -> 85.77 | Improved |
| add last-layer attention separation loss L_AS^la (ablation) | ResNet-50 + L_AS^in + L_AC · CUB-200-2011 | top-1 accuracy: 85.77 -> 86.20 | Improved |
| use channel-weighted attention A_ch instead of Grad-CAM inside ICASC (ablation) | ICASC · CUB-200-2011 | top-1 accuracy: 85.45 -> 86.20 | Improved |
| use channel-weighted attention A_ch instead of Grad-CAM inside ICASC (ablation) | ICASC · Caltech-256 (N=60 per class) | top-1 accuracy: 80.28 -> 81.32 | Improved |
| use channel-weighted attention A_ch instead of Grad-CAM inside ICASC (ablation) | ICASC · ILSVRC2012 val (single crop) | top-1 accuracy: 69.84 -> 69.90 | No clear effect |
| use channel-weighted attention A_ch instead of Grad-CAM inside ICASC (ablation) | ICASC · PASCAL VOC 2012 classification | AP (%): 82.12 -> 83.17 | Improved |
| use A_ch instead of Grad-CAM++ as weak localization cue (ablation) | DeepLab on VGG-19 attention · PASCAL VOC 2012 seg val (weak supervision) | segmentation score: 51.70 -> 57.97 | Improved |
| use A_ch instead of Grad-CAM as weak localization cue (ablation) | DeepLab on VGG-19 attention · PASCAL VOC 2012 seg val (weak supervision) | segmentation score: 56.65 -> 57.97 | Improved |
| add ICASC training with Grad-CAM attention | ResNet-110 · CIFAR-100 | top-1 accuracy: 72.78 -> 74.02 | Improved |
| add ICASC training with A_ch attention | ResNet-110 · CIFAR-100 | top-1 accuracy: 72.78 -> 76.11 | Improved |
| add ICASC training with A_ch attention | ResNet-18 · ILSVRC2012 val (single crop) | top-1 accuracy: 69.51 -> 69.90 | No clear effect |
Closest papers by meaning
- Learn to Pay Attention (ICLR 2018 · similarity 0.90)
- Explicitly Modeled Attention Maps for Image Classification (AAAI 2021 · similarity 0.89)
- One Explanation is Not Enough: Structured Attention Graphs for Image Classification (NeurIPS 2021 · similarity 0.88)
- Exploring Self-Attention for Image Recognition (CVPR 2020 · similarity 0.87)
- AttentionRNN: A Structured Spatial Attention Mechanism (ICCV 2019 · similarity 0.87)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
