Research map / Architectures and efficiency / Attention and transformers
Compressed Self-Attention for Deep Metric Learning
Ziye Chen, Mingming Gong, Yanwu Xu, Chaohui Wang, Kun Zhang, Bo Du
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CSA modules | PCB · Market-1501 | mAP: 77.4 -> 82.3 | Improved |
| add CSA modules | PCB · DukeMTMC-reID | mAP: 66.1 -> 73.5 | Improved |
| add CSA modules | PCB · CUHK03-NP | mAP: 53.2 -> 63.7 | Improved |
| add CSA modules | PCB-RPP · Market-1501 | mAP: 81.6 -> 83.5 | Improved |
| add CSA modules | PCB-RPP · CUHK03-NP | mAP: 56.7 -> 65.0 | Improved |
| replace original self-attention with CSA (with channel grouping) (ablation) | PCB · CUHK03-NP | mAP: 62.7 -> 63.7 | Improved |
| increase base attention maps per group from 8 to 32 (ablation) | PCB + CSA · Market-1501 | mAP: 82.1 -> 82.3 | Improved |
| increase base attention maps per group from 32 to 64 (ablation) | PCB + CSA · Market-1501 | mAP: 82.3 -> 82.2 | Hurt |
| increase number of groups from 1 to 2 (ablation) | PCB + CSA · Market-1501 | mAP: 81.9 -> 82.3 | Improved |
| increase number of groups from 4 to 8 (ablation) | PCB + CSA · DukeMTMC-reID | mAP: 73.4 -> 73.3 | Hurt |
| place CSA on 1st, 2nd, 3rd stages instead of only 1st stage (ablation) | PCB + CSA · Market-1501 | mAP: 81.6 -> 82.3 | Improved |
| add deep supervision to CSA modules (ablation) | PCB + CSA · Market-1501 | mAP: 81.3 -> 82.3 | Improved |
Closest papers by meaning
- Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Loss (ECCV 2020 · similarity 0.89)
- On the Integration of Self-Attention and Convolution (CVPR 2022 · similarity 0.88)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.86)
- Self-Attention-Based Contextual Modulation Improves Neural System Identification (ICLR 2025 · similarity 0.85)
- Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency (NeurIPS 2025 · similarity 0.85)
- Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
