Research map / Video / Action recognition
Relational Self-Attention: What's Missing in Attention for Video Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add relational kernel to the basic kernel (ablation) | RSA · SS-V1 | top-1 accuracy: 45.7 -> 47.0 | Helped |
| Add relational context to the basic context (ablation) | RSA · SS-V1 | top-1 accuracy: 45.7 -> 47.0 | Helped |
| Use relational kernel instead of basic kernel with basic context (ablation) | RSA · SS-V1 | top-1 accuracy: 44.8 -> 45.4 | Helped |
| Use both basic and relational kernels with basic context (ablation) | RSA · SS-V1 | top-1 accuracy: 45.4 -> 45.7 | Helped |
| Use relational context with a basic kernel instead of basic context (ablation) | RSA · SS-V1 | top-1 accuracy: 44.8 -> 46.5 | Helped |
| Increase kernel size from 3 × 3 × 3 to 5 × 7 × 7 (ablation) | RSA · SS-V1 | top-1 accuracy: 49.4 -> 51.5 | Helped |
| Increase kernel size from 5 × 7 × 7 to 5 × 9 × 9 (ablation) | RSA · SS-V1 | top-1 accuracy: 51.5 -> 51.2 | Hurt |
| Use full Hadamard channel-wise correlation instead of scalar dot-product correlation (ablation) | RSA · SS-V1 | top-1 accuracy: 50.4 -> 51.5 | Helped |
| Use more queries, increasing L from 2 to 8 (ablation) | RSA · SS-V1 | top-1 accuracy: 50.6 -> 51.3 | Helped |
| Increase query count from 8 to 16 (ablation) | RSA · SS-V1 | top-1 accuracy: 51.3 -> 50.0 | Hurt |
| Replace more spatial convolution layers with RSA layers (ablation) | TSN-ResNet50 · SS-V1 | top-1 accuracy: 47.0 -> 51.5 | Helped |
| Apply L2 normalization to query, key, and value (ablation) | RSA · SS-V1 | top-1 accuracy: 50.3 -> 51.5 | Helped |
Closest papers by meaning
- SSAN: Separable Self-Attention Network for Video Representation Learning (CVPR 2021 · similarity 0.88)
- Is Space-Time Attention All You Need for Video Understanding? (ICML 2021 · similarity 0.87)
- Learning Correlation Structures for Vision Transformers (CVPR 2024 · similarity 0.87)
- Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis (NeurIPS 2024 · similarity 0.85)
- Graph Convolutions Enrich the Self-Attention in Transformers! (NeurIPS 2024 · similarity 0.85)
- Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.