Research map / Large language models / Language, safety and interpretability
Language-Agnostic Representation Learning of Source Code from Structure and Context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine source-code context with AST structure (ablation) | Code Transformer · Java-small | F1: 50.34 -> 52.22 | Helped |
| Combine source-code context with AST structure (ablation) | Code Transformer · CSN Ruby | micro F1: 27.72 -> 27.50 | No clear effect |
| Train jointly on four programming languages rather than one | Code Transformer · CSN Python | micro F1: 34.97 -> 36.18 | Helped |
| Train jointly on four programming languages rather than one | Code Transformer · CSN Ruby | micro F1: 27.50 -> 31.24 | Helped |
| Use multilingual training without AST structure (ablation) | Code Transformer · CSN Python | micro F1: 36.18 -> 33.80 | Hurt |
| Pre-train on multilingual masked language modeling before task fine-tuning | Code Transformer · CSN Python | micro F1: 36.18 -> 37.35 | Helped |
| Pre-train on Java-medium and Java-large before Java-small fine-tuning | Code Transformer · Java-small | F1: 52.22 -> 53.77 | Helped |
| Remove the pointer network (ablation) | Code Transformer · CSN Go | micro F1: 51.34 -> 52.13 | Helped |
| Remove the pointer network (ablation) | Code Transformer · CSN Python | micro F1: 34.55 -> 34.97 | Hurt |
| Combine sibling, ancestor, shortest-path, and Personalized PageRank AST distances (ablation) | Structure-only Code Transformer without pointer network · Java-small | F1: 48.47 -> 48.75 | Helped |
| Use 16 or 64 bins instead of 32 (ablation) | Code Transformer · Code summarization | performance | No clear effect |
| Fine-tune multilingual training on individual languages | Code Transformer · CSN | micro F1: 36.18 -> 35.98 | No clear effect |
Closest papers by meaning
- MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning (AAAI 2023 · similarity 0.91)
- Learning and Evaluating Contextual Embedding of Source Code (ICML 2020 · similarity 0.89)
- CODE REPRESENTATION LEARNING AT SCALE (ICLR 2024 · similarity 0.87)
- AST-T5: Structure-Aware Pretraining for Code Generation and Understanding (ICML 2024 · similarity 0.87)
- CONSIDER: Commonalities and Specialties Driven Multilingual Code Retrieval Framework (AAAI 2024 · similarity 0.86)
- ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.