Research map / Architectures and efficiency / Attention and transformers
CLUENet: Cluster Attention Makes Neural Networks Have Eyes
Xiangshuai Song, Jun-Jie Huang, Tianrui Liu, Ke Liang, Chang Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add global soft feature aggregation (incl. TCosAttn and gate) (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 76.97 -> 78.75 | Improved |
| use temperature-scaled cosine attention instead of standard attention (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): -0.59 Top-1 when removed (text) | Improved |
| add gated residual fusion (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.46 -> 78.75 | Improved |
| add shared dispatching across blocks (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.32 -> 78.75 | Improved |
| add learnable positional embedding (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.21 -> 78.75 | Improved |
| replace FEC cluster pooling with improved cluster pooling (pooling in similarity space, 2-layer MLP) (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.07 -> 78.75 | Improved |
| use 2-layer MLP instead of single layer for projv (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.10 -> 78.75 | Improved |
| increase projv depth from two to three layers (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.75 -> 78.71 | No clear effect |
| add cluster-center-guided semantic query (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 77.72 -> 78.75 | Improved |
| use multi-head instead of single-head center query (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.41 -> 78.75 | Improved |
| use global clustering with 49 clusters instead of window partition (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.30 -> 78.75 | Improved |
| increase cluster number from 49 to 64 (ablation) | CLUENet (micro) · Mini-ImageNet val | Top-1 accuracy (%): 78.75 -> 78.70 | No clear effect |
Closest papers by meaning
- Network Dissection: Quantifying Interpretability of Deep Visual Representations (CVPR 2017 · similarity 0.87)
- PICNN: A Pathway towards Interpretable Convolutional Neural Networks (AAAI 2024 · similarity 0.87)
- ClusterGNN: Cluster-Based Coarse-To-Fine Graph Neural Network for Efficient Feature Matching (CVPR 2022 · similarity 0.87)
- Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models (CVPR 2025 · similarity 0.86)
- Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels (ICLR 2024 · similarity 0.86)
- Revisiting the Integration of Convolution and Attention for Vision Backbone (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
