Research map / Architectures and efficiency / Attention and transformers
Quaternion Self-Attention with Shared Scores
Shogo Yamauchi, Tohru Nitta, Hideaki Tamori
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Hamilton component-wise attention with shared-score attention | QTransformer · DNS-Challenge 3 | GPU RTF: 0.0192 -> 0.0107 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QTransformer · DNS-Challenge 3 | CPU RTF: 0.594 -> 0.249 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QConformer · DNS-Challenge 3 | GPU RTF: 0.0202 -> 0.0157 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QConformer · DNS-Challenge 3 | CPU RTF: 0.610 -> 0.259 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QTransformer · DNS-Challenge 3 | DNSMOS OVRL: 2.61 -> 2.67 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QConformer · VoiceBank+DEMAND | PESQ: 3.11 -> 3.18 | Improved |
| replace Hamilton component-wise attention with shared-score attention | QConformer · VoiceBank+DEMAND | SI-SDR: 19.64 -> 19.36 | No clear effect |
| replace Hamilton component-wise attention with shared-score attention | QTransformer · VoiceBank+DEMAND | PESQ: 3.07 -> 3.07 | No clear effect |
| replace Hamilton component-wise attention with shared-score attention | quaternion Transformer · CIFAR-100 | accuracy (%): 71.09 -> 70.94 | No clear effect |
| replace Hamilton component-wise attention with shared-score attention | quaternion Transformer · SST-2 | accuracy (%): 81.12 -> 81.04 | No clear effect |
| add attention bottleneck (QTransformer/QConformer) to QDenseNet (ablation) | quaternion SE model · VoiceBank+DEMAND | PESQ: 2.80 -> 3.18 | Improved |
| use Hamilton-product QTransformer attention instead of QConv-based attention (QTN) (ablation) | quaternion SE model · VoiceBank+DEMAND | PESQ: 2.76 -> 3.07 | Improved |
Closest papers by meaning
- Prompt Learning with Quaternion Networks (ICLR 2024 · similarity 0.86)
- Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency (NeurIPS 2025 · similarity 0.84)
- Theory, Analysis, and Best Practices for Sigmoid Self-Attention (ICLR 2025 · similarity 0.84)
- Self-Attention-Based Contextual Modulation Improves Neural System Identification (ICLR 2025 · similarity 0.83)
- Customizing the Inductive Biases of Softmax Attention using Structured Matrices (ICML 2025 · similarity 0.83)
- Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
