Research map / Architectures and efficiency / Attention and transformers
Representational Strengths and Limitations of Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Closest papers by meaning
- On the Role of Attention Masks and LayerNorm in Transformers (NeurIPS 2024 · similarity 0.89)
- Transformers Learn Faster with Semantic Focus (NeurIPS 2025 · similarity 0.89)
- Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) (AAAI 2024 · similarity 0.89)
- What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis (ICLR 2025 · similarity 0.88)
- When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective (NeurIPS 2025 · similarity 0.88)
- What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.