Research map / Architectures and efficiency / RNNs and normalization
SemRep : Generative Code Representation Learning with Code Transformations
Weichen Li, Jiamin Song, Bogdan Alexandru Stoica, Arav Dhoot, Gabriel Ryan, Shengyu Fu, Kexin Pei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add generative code representation learning (mid-training) (ablation) | finetuned QwQ-32B · KernelBench | Correctness avg@16: 12.56 -> 25.75 | Improved |
| add generative code representation learning (mid-training) (ablation) | finetuned QwQ-32B · EditBench Core | Pass@16 (%): 55.56 -> 62.96 | Improved |
| add SemRep test-time scaling to finetuned baseline (ablation) | finetuned QwQ-32B · EditBench Core | Pass@1 (%): 53.70 -> 56.48 | Improved |
| add SemRep test-time scaling to finetuned baseline (ablation) | finetuned QwQ-32B · KernelBench | Correctness best@16: 74 -> 59 | Hurt |
| add SemRep test-time scaling to finetuned baseline (ablation) | finetuned QwQ-32B · KernelBench | Speedup best@16: 1.33 -> 1.47 | Improved |
| combine SemRep training and TTS (ablation) | finetuned QwQ-32B · KernelBench | Correctness best@16: 74 -> 93 | Improved |
| combine SemRep training and TTS (ablation) | finetuned QwQ-32B · KernelBench | Speedup best@16: 1.33 -> 2.87 | Improved |
| combine SemRep training and TTS (ablation) | finetuned QwQ-32B · EditBench Core | Pass@16 (%): 55.56 -> 66.67 | Improved |
| add TTS on top of SemRep-trained model (ablation) | SemRep-trained QwQ-32B · KernelBench | Speedup avg@16: 1.25 -> 1.21 | Hurt |
| use SemRep instead of finetuning only | QwQ-32B · KernelBench on unseen H200 | Correctness avg@16: 12.63 -> 20.38 | Improved |
| use SemRep instead of finetuning only | QwQ-32B · EditBench Core (perturbed) | Consistency (%): 75.00 -> 88.89 | Improved |
| use SemRep instead of Kevin-32B | QwQ-32B · KernelBench (100 test tasks) | Correctness best@16: 65 -> 93 | Improved |
Closest papers by meaning
- CODE REPRESENTATION LEARNING AT SCALE (ICLR 2024 · similarity 0.88)
- Training Language Models on Synthetic Edit Sequences Improves Code Synthesis (ICLR 2025 · similarity 0.87)
- Towards A Generalist Code Embedding Model Based On Massive Data Synthesis (NeurIPS 2025 · similarity 0.87)
- Exploiting Code Symmetries for Learning Program Semantics (ICML 2024 · similarity 0.87)
- EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning (ICML 2025 · similarity 0.87)
- SelfCodeAlign: Self-Alignment for Code Generation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
