Research map / Large language models / Agents, code and math
CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CodeRosetta, a 0.8B encoder-decoder transformer trained unsupervised (cross-lingual MLM, AST entity recognition, tailored denoising, back-translation, optional synthetic-data fine-tuning) for C++↔CUDA and Fortran↔C++ translation.
Problem
Unsupervised code translation between a language and its HPC parallel extensions (C++ to CUDA, Fortran to C++) is underexplored due to scarce parallel corpora and complex parallel semantics; prior models were unidirectional or relied on language-specific metrics.
Approach
An encoder-decoder transformer with pretraining and denoising objectives tailored to HPC extensions: cross-lingual masked LM, AST entity recognition (tree-sitter tags), weighted token dropping prioritizing language keywords, foreign-token insertion, and an adaptive noise rate, plus back-translation and optional LLM-synthetic fine-tuning.
Key results
- On C++→CUDA, CodeRosetta reaches 76.90 BLEU and 78.84 CodeBLEU, versus 74.00 and 77.12 for BabelTower, with 98.85% compilation accuracy after post-processing.
- On Fortran→C++, CodeRosetta (0.8B) reaches 65.93 CodeBLEU versus 61.30 for fine-tuned StarCoder (15B) and 19.21 for GPT-4.
- Ablation: removing MLM drops C++→CUDA CodeBLEU by 26.88 points; removing fine-tuning drops it by 7.63.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MLM pretraining (ablation) | CodeRosetta · C++ to CUDA | BLEU: 52.12 -> 76.90 | Helped |
| Add AER pretraining (ablation) | CodeRosetta · C++ to CUDA | CodeBLEU: 75.55 -> 78.84 | Helped |
| Use DAE special noises (ablation) | CodeRosetta · C++ to CUDA | CodeBLEU: 73.22 -> 78.84 | Helped |
| Add back translation (ablation) | CodeRosetta · C++ to CUDA | CodeBLEU: 73.18 -> 78.84 | Helped |
| Fine-tune on synthetic data (ablation) | CodeRosetta · C++ to CUDA | CodeBLEU: 71.21 -> 78.84 | Helped |
| Use CodeRosetta instead of BabelTower | C++ to CUDA · C++ to CUDA | BLEU: +2.9 BLEU | Helped |
| Use CodeRosetta instead of BabelTower | C++ to CUDA · C++ to CUDA | CodeBLEU: +1.72 points | Helped |
| Use CodeRosetta instead of BabelTower | C++ to CUDA · C++ to CUDA | compilation accuracy: 6.05% | Helped |
| Use CodeRosetta instead of fine-tuned StarCoder | Fortran to C++ · Fortran to C++ | CodeBLEU: 61.30 -> 65.93 | Helped |
| Use CodeRosetta instead of GPT-4 | CUDA to C++ · CUDA to C++ | BLEU: 70.18 -> 77.03 | Helped |
| Use CodeRosetta instead of GPT-4 | C++ to Fortran · C++ to Fortran | CodeBLEU: 35.32 -> 70.46 | Helped |
Surprising findings
- Beam size had little effect: beam sizes 1 to 50 changed BLEU by under 0.5 points, and the top output was largely unchanged.
- Gemini-Pro and Gemini-Ultra frequently produced C++ nearly identical to CUDA inputs when translating CUDA to C++.
Limitations
- Compilation accuracy relies on manual post-processing fixes (generic type definitions, missing initializations, closing braces) rather than automated compiler feedback.
- Functional correctness checked only on 30 sampled CUDA kernels, and the Fortran→C++ evaluation uses only 33 test pairs.
- BabelTower and TransCoder results are quoted from their paper because their models were unavailable, so comparisons rest on reported numbers.
Experimental setup
Datasets: BabelTower C++-CUDA dataset Stack V2 Bin et al. paired C++-Fortran dataset synthetic C++-CUDA pairs from GPT-4 and Gemini
Benchmarks: BabelTower C++-CUDA test set (180 pairs) Bin et al. C++-Fortran test set (33 pairs)
Metrics: BLEU CodeBLEU ChrF ROUGE-L compilation accuracy functional correctness (30 kernels)
Baselines: BabelTower TransCoder GPT-4 Gemini-Ultra Gemini-Pro StarCoder2-7B DeepSeek-Coder-V2-Lite-Base StarCoder (fine-tuned, Bin et al.)
Builds on: TransCoder BabelTower UniXcoder CodeT5
Closest papers by meaning
- QiMeng-MuPa: Mutual-Supervised Learning for Sequential-to-Parallel Code Translation (NeurIPS 2025 · similarity 0.90)
- CUDABench: Benchmarking LLMs for Text-to-CUDA Generation (NeurIPS 2026 · similarity 0.85)
- TorchTitan: One-stop PyTorch native solution for production ready LLM pretraining (ICLR 2025 · similarity 0.84)
- McEval: Massively Multilingual Code Evaluation (ICLR 2025 · similarity 0.84)
- Can Large Language Models Understand Intermediate Representations in Compilers? (ICML 2025 · similarity 0.84)
- CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.