Research map / Architectures and efficiency / Attention and transformers
A STRUCTURED SELF-ATTENTIVE SENTENCE EMBEDDING
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use structured self-attention sentence embeddings instead of max-pooled sentence encodings | Our Model · Yelp | accuracy: 61.99% -> 64.21% | Helped |
| use structured self-attention sentence embeddings instead of max-pooled sentence encodings | Our Model · Age | accuracy: 77.40% -> 80.45% | Helped |
| use structured self-attention sentence embeddings | SNLI sentence encoding models · SNLI test | accuracy: 83.4% -> 84.4%; 84.6% -> 84.4% | Mixed |
| add the attention-diversity penalization term (ablation) | Our Model · Yelp | accuracy: 61.74% -> 64.21% | Helped |
| add the attention-diversity penalization term (ablation) | Our Model · Age | accuracy: 79.27% -> 80.45% | Helped |
| add the attention-diversity penalization term (ablation) | Our Model · SNLI | accuracy | No clear effect |
| use multiple attention vectors instead of a single sentence vector (ablation) | Our Model · Age | accuracy | Helped |
| use multiple attention vectors instead of a single sentence vector (ablation) | Our Model · SNLI | accuracy | Helped |
| vary the number of attention rows within the reported range of 10 to 30 (ablation) | Our Model · Age | accuracy | No clear effect |
| vary the number of attention rows within the reported range of 10 to 30 (ablation) | Our Model · SNLI | accuracy | No clear effect |
| prune the hidden-layer weight connections using the matrix embedding structure (ablation) | Our Model · Yelp | accuracy: 64.21% -> 63.86% | Hurt |
| prune the hidden-layer weight connections using the matrix embedding structure (ablation) | Our Model · Age | accuracy: 80.45% -> 77.32% | Hurt |
Closest papers by meaning
- No Training Required: Exploring Random Encoders for Sentence Classification (ICLR 2019 · similarity 0.87)
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.87)
- Scalable Attentive Sentence Pair Modeling via Distilled Sentence Embedding (AAAI 2020 · similarity 0.87)
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks (ICLR 2017 · similarity 0.86)
- LLM-based Embeddings: Attention Values Encode Sentence Semantics Better Than Hidden States (ICML 2026 · similarity 0.86)
- Multiple Positional Self-Attention Network for Text Classification (AAAI 2020 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.