Research map / Architectures and efficiency / Attention and transformers
Integrating Tree Path in Transformer for Code Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add relative path encoding in Key and Value (ablation) | TPTrans Large · CodeSearchNet Python | F1: 31.42 -> 36.59 | Helped |
| add relative path encoding in Key and Value (ablation) | TPTrans Large · CodeSearchNet Ruby | F1: 22.33 -> 29.82 | Helped |
| add relative path in Value (on top of Key) (ablation) | TPTrans Large · CodeSearchNet Ruby | F1: 27.76 -> 29.82 | Helped |
| add absolute path encoding to vanilla Transformer (ablation) | Transformer · CodeSearchNet Python | F1: 31.42 -> 33.70 | Helped |
| add absolute path on top of full relative path encoding (ablation) | TPTrans Large · CodeSearchNet (4 languages) | F1: Python 36.59 vs 36.45 | No clear effect |
| use full path node information instead of hops only (ablation) | TPTrans Base · CodeSearchNet Ruby | F1: 27.05 -> 30.02 | Helped |
| use full path node information instead of hops only (ablation) | TPTrans Base · CodeSearchNet Python | F1: 34.89 -> 35.88 | Helped |
| add normalization of GRU path encoding (ablation) | TPTrans Base · CodeSearchNet Ruby | F1: 24.05 -> 30.02 | Helped |
| add normalization of GRU path encoding (ablation) | TPTrans Base · CodeSearchNet Python | F1: 34.74 -> 35.88 | Helped |
| add path encoding to deeper 6-layer Transformer (ablation) | TPTrans deep · CodeSearchNet Ruby | F1: 21.71 -> 28.82 | Helped |
| TPTrans vs Code Transformer baseline | TPTrans Base · CodeSearchNet Javascript | F1: 32.13 -> 30.65 | Hurt |
| TPTrans vs Code Transformer baseline | TPTrans Base · CodeSearchNet Go | F1: 51.33 -> 53.54 | Helped |
Closest papers by meaning
- PaTH Attention: Position Encoding via Accumulating Householder Transformations (NeurIPS 2025 · similarity 0.86)
- AST-T5: Structure-Aware Pretraining for Code Generation and Understanding (ICML 2024 · similarity 0.84)
- Distilling Sequential Computation in Transformer Language Models (NeurIPS 2026 · similarity 0.84)
- JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.83)
- Effective Context in Transformers: An Analysis of Fragmentation and Tokenization (NeurIPS 2026 · similarity 0.83)
- How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.