Research map / Architectures and efficiency / Attention and transformers
Weight Excitation: Built-in Attention Mechanisms in Convolutional Neural Networks
Niamul Quader, Md Mafijul Islam Bhuiyan, Juwei Lu, Peng Dai, Wei Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add location-based weight excitation (LWE) | ResNet50 · ImageNet val | top-1 accuracy (%): 75.8 -> 77.1 | Improved |
| add magnitude-based weight excitation (MWE) (ablation) | ResNet50 · ImageNet val | top-1 accuracy (%): 75.83 -> 76.52 | Improved |
| add LWE instead of MWE (ablation) | ResNet50 · ImageNet val | top-1 accuracy (%): 76.52 -> 77.07 | Improved |
| combine LWE with MWE (ablation) | ResNet50 · ImageNet val | top-1 accuracy (%): 77.07 -> 77.09 | No clear effect |
| add LWE on top of SE (ablation) | ResNet50-SE · ImageNet val | top-1 accuracy (%): 77.1 -> 77.5 | Improved |
| add LWE on top of SE (ablation) | ResNet152-SE · ImageNet val | top-1 accuracy (%): 78.7 -> 79.0 | Improved |
| add LWE to depthwise-conv network | MobileNetV2 · ImageNet val | top-1 accuracy (%): 66.2 -> 66.5 | Mixed |
| use MWE on depthwise convolutions instead of LWE (ablation) | MobileNetV2 · ImageNet val | top-1 accuracy (%): 66.5 -> 67.0 | Improved |
| add LWE vs weight standardization/normalization (ablation) | ResNet50 · ImageNet val | top-1 accuracy (%): 76.1 -> 77.1 | Improved |
| add spectral-norm style reparameterization (SN) (ablation) | ResNet50 · ImageNet val | top-1 accuracy (%): 75.8 -> 75.7 | No clear effect |
| train LWE for 90 epochs instead of 100 (ablation) | ResNet50-LWE · ImageNet val | top-1 accuracy (%): 77.1 -> 76.9 | No clear effect |
| set MWE epsilon_A to 10^-3 / 0.1 / 0.2 (ablation) | ResNet50-MWE · ImageNet val | top-1 accuracy (%): 76.38 / 76.52 / 76.43 | No clear effect |
Closest papers by meaning
- WeightNet: Revisiting the Design Space of Weight Networks (ECCV 2020 · similarity 0.91)
- SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks (ICML 2021 · similarity 0.87)
- Squeeze-and-Excitation Networks (CVPR 2018 · similarity 0.86)
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs (ICLR 2024 · similarity 0.85)
- LSNet: See Large, Focus Small (CVPR 2025 · similarity 0.85)
- Pay Attention to Small Weights (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
