Research map / Large language models / AI and society
CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks
Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti and 5 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Retain token order with a CNN instead of using a bag of tokens | Code classification · C++1400 | accuracy: 64.50±0.13 -> 93.71±0.18 | Improved |
| Add a virtual node to the SPT graph (ablation) | GCN code classifier · C++1400 | accuracy: 95.26±0.13 -> 95.73±0.07 | Improved |
| Add a virtual node to the SPT graph (ablation) | GIN code classifier · C++1400 | accuracy: 95.95±0.13 -> 96.36±0.10 | Improved |
| Increase GNN depth beyond 5 layers (ablation) | SPT code classification · CodeNet benchmarks | accuracy | No clear effect |
| Use code graph representation instead of SPT (ablation) | GCN code classifier · Java250 | accuracy: 92.70±0.25 -> 94.10±.001 | Improved |
| Use code graph representation instead of SPT (ablation) | GCN code classifier · Python800 | accuracy: 93.82±0.16 -> 87.80±.007 | Hurt |
| Retain token order and use a Siamese network instead of bag-of-tokens MLP | Code similarity · C++1400 | accuracy: 86.54±0.07 -> 96.56±0.07 | Improved |
| Use GMN cross-graph attention instead of MISIM structure-only GNN (ablation) | Java250 similarity · Java250 (p4, s5) | MAP@R: 0.472±0.023 -> 0.679±0.056 | Improved |
| Add MISIM node attributes to GMN (ablation) | Java250 similarity GMN · Java250 (p4, s5) | MAP@R: 0.679±0.056 -> 0.985±0.015 | Improved |
| Train MISIM on cleaned and de-duplicated C++1000 instead of GCJ-297 | MISIM · POJ-104 test set | MAP@R: 12% higher | Improved |
| Train MISIM on C++1000 instead of GCJ-297 | MISIM · GCJ-297 validation set | MAP@R: 10% higher validation score with GCJ-297 | Hurt |
| Remove one of two similar problems from C++ classification data (ablation) | C-BERT · C++ classification | accuracy | Improved |
Closest papers by meaning
- JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.88)
- CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs (ICLR 2025 · similarity 0.87)
- Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models (AAAI 2024 · similarity 0.87)
- RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph (ICLR 2025 · similarity 0.87)
- CODE REPRESENTATION LEARNING AT SCALE (ICLR 2024 · similarity 0.87)
- Towards A Generalist Code Embedding Model Based On Massive Data Synthesis (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
