Research map / Learning theory and algorithms / Kernels and regression theory
Kernel Multimodal Continuous Attention
Alexander Moreno, Zhenke Wu, Supriya Nagesh, Walter H. Dempsey, James Matthew Rehg
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Gaussian/unimodal continuous attention with kernel exponential family attention | continuous softmax attention · Synthetic time warping | accuracy: 77.72 -> 85.71 | Improved |
| replace truncated-parabola continuous sparsemax with kernel deformed exponential attention | continuous sparsemax · Synthetic time warping | accuracy: 77.96 -> 88.1 | Improved |
| use kernel sparsemax instead of continuous sparsemax | continuous sparsemax · FordA | accuracy: 90.95 -> 92.61 | Improved |
| use kernel softmax instead of Gaussian-based continuous softmax | continuous softmax · FordA | accuracy: 74.88 -> 92.44 | Improved |
| use kernel softmax instead of Gaussian mixture attention | Gaussian mixture · FordA | accuracy: 69.56 -> 92.44 | Improved |
| kernel attention compared with LSTM FCN | kernel sparsemax · FordA | accuracy: 93.41 vs 92.61 | Hurt |
| use kernel softmax instead of multi-head unimodal continuous softmax (100 heads) | continuous softmax · uWave irregular sampling | accuracy: 67.78 -> 93.85 | Improved |
| use kernel softmax instead of Gaussian mixture attention | Gaussian mixture · uWave irregular sampling | accuracy: 81.13 -> 94.26 | Improved |
| increase basis functions N from 64 to 128 | kernel softmax · uWave irregular sampling | accuracy: 93.85 -> 94.26 | No clear effect |
| increase basis functions N from 64 to 128 | kernel sparsemax · uWave irregular sampling | accuracy: 92.32 -> 92.15 | No clear effect |
| use kernel softmax instead of continuous softmax | continuous softmax · IMDB | mean accuracy: 90.32 -> 90.94 | Improved |
| use kernel sparsemax instead of continuous sparsemax | continuous sparsemax · IMDB | mean accuracy: 89.95 -> 90.39 | Improved |
Closest papers by meaning
- Sparse Attention as Compact Kernel Regression (NeurIPS 2026 · similarity 0.87)
- Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems (NeurIPS 2025 · similarity 0.83)
- Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes (ICML 2024 · similarity 0.82)
- Scale-invariant attention (NeurIPS 2025 · similarity 0.82)
- Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers (NeurIPS 2024 · similarity 0.82)
- MultiMax: Sparse and Multi-Modal Attention Learning (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
