Research map / Architectures and efficiency / Attention and transformers
A Convolutional Attention Network for Extreme Summarization of Source Code
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add copy attention | Convolutional Attention · 10 Java projects, test sets | F1 rank 1 (%): 43.6 -> 44.7 | Helped |
| Add copy attention | Convolutional Attention · 10 Java projects, test sets | F1 rank 5 (%): 57.7 -> 59.6 | Helped |
| Use convolutional attention instead of standard attention | code summarization · 10 Java projects, test sets | F1 rank 1 (%): 33.6 -> 43.6 | Helped |
| Use convolutional attention instead of standard attention | code summarization · 10 Java projects, test sets | F1 rank 5 (%): 45.2 -> 57.7 | Helped |
| Use the convolutional copy model instead of tf-idf | code summarization · 10 Java projects, test sets | Exact match rank 1 (%): 24.3 -> 23.5 | Hurt |
| Use the convolutional copy model instead of tf-idf | code summarization · 10 Java projects, test sets | F1 rank 1 (%): 40.0 -> 44.7 | Helped |
| Shuffle the input subtokens, removing sequential features (ablation) | Copy Convolutional Attention · libgdx | F1 rank 1 (%): F1 at rank 1 that is +1% compared to tf-idf | Hurt |
| Shuffle the input subtokens, removing sequential features (ablation) | Standard Attention · libgdx | F1 rank 1 (%): 41.8% -> 26.2% | Hurt |
| Replace the GRU state with embeddings of the previous two subtokens (ablation) | Copy Convolutional Attention · 10 Java projects, test sets | F1: reducing F1 by less than 1% | No clear effect |
| Add copying from input tokens | code summarization · out-of-vocabulary subtokens | OoV accuracy rank 1 (%): 10.5% | Helped |
Closest papers by meaning
- Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.84)
- SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking (NeurIPS 2026 · similarity 0.84)
- Training-Free Long-Context Scaling of Large Language Models (ICML 2024 · similarity 0.84)
- Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models (NeurIPS 2026 · similarity 0.83)
- Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study (ICLR 2025 · similarity 0.83)
- Customizing the Inductive Biases of Softmax Attention using Structured Matrices (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.