Research map / Large language models / Agents, code and math
Function-to-Style Guidance of LLMs for Code Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes F2STrans, a two-stage function-then-style fine-tuning paradigm for LLM code translation, plus a new Codeforces-based benchmark with manually annotated ground-truth translations and extensive tests.
Problem
LLM code translation suffers from both functional incorrectness and poor readability (failure to preserve source style such as variable naming and structure), and existing benchmarks are outdated, have few tests, and lack ground-truth translations.
Approach
Mine function-consistent cross-language solution pairs from online platforms (embedding retrieval, LLM-judge relevance scoring, differential testing) for functional SFT, then apply a list-wise contrastive style loss over style-consistent positive translations (selected by CSSim consensus) and style-inconsistent negatives generated by the functional model.
Key results
- On the F2STrans Benchmark, Qwen-1.5B with F2STrans averages 69.8 CA, surpassing Qwen-32B direct prompting (53.9) and GPT-4 self-debug (68.5) on average across 20 scenarios.
- On CodeNet, F2STrans Qwen-0.5B averages 78.9 CA versus 75.4-ish for RAG Qwen-32B and 81.9 for GPT-4 self-debug; StarCoder-3B reaches 86.7 average on CodeNet with the method.
- Style evaluation: F2STrans Qwen-0.5B achieves CSSim 80.7 vs 79.5 for GPT-4 self-debug.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply function-to-style guidance rather than GPT-4 self-debug prompting | Qwen7B · CodeNet | CA: 81.9 -> 89.8 | Helped |
| Apply function-to-style guidance rather than GPT-4 self-debug prompting | Qwen7B · F2STrans benchmark | CA: 68.5 -> 75.9 | Helped |
| Apply function-to-style guidance rather than GPT-4 self-debug prompting | Qwen7B · xCodeEval | CA: 82.0 -> 89.1 | Helped |
| Add relevance-driven code pair selection (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 75.5 -> 78.9; 57.5 -> 62.1 | Helped |
| Add the LLM judge to code pair selection (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 77.5 -> 78.9; 59.5 -> 62.1 | Helped |
| Add differential testing (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 78.2 -> 78.9; 60.2 -> 62.1 | Helped |
| Add style-aware information to the prompt (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 78.6 -> 78.9; 61.5 -> 62.1 | Helped |
| Select positive translations by style consensus instead of randomly (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 77.7 -> 78.9; 58.6 -> 62.1 | Helped |
| Select style-poor negative translations instead of randomly (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 78.4 -> 78.9; 60.3 -> 62.1 | Helped |
| Add the list-wise loss (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 74.3 -> 78.9; 56.8 -> 62.1 | Helped |
| Add positive-example instruction-tuning loss (ablation) | F2STrans with Qwen0.5B · CodeNet and F2STrans benchmark | CA: 75.9 -> 78.9; 59.0 -> 62.1 | Helped |
Surprising findings
- Qwen-0.5B with F2STrans outperforms GPT-4 self-debug in style similarity despite lower functional accuracy.
- Without the function and style stages, base Qwen-0.5B and StarCoder-3B score below 10 CA on CodeNet-style evaluation.
Limitations
- Benchmark accuracy on Python translation is much lower and attributed speculatively to interpreted nature.
- Style guidance relies on CSSim, which is limited to variable naming, API, and structure, and on a 32B model for positive translations.
Experimental setup
Datasets: CodeNet xCodeEval Codeforces solutions (mined)
Benchmarks: CodeNet F2STrans Benchmark xCodeEval
Metrics: Computational Accuracy CSSim Dis_var Dis_api Dis_stru
Baselines: Direct prompting Chain-of-thought prompting RAG prompting Self-debug DPO
Builds on: CSSim Jina code embeddings DPO self-debugging
Closest papers by meaning
- Automatically Testing Functional Properties of Code Translation Models (AAAI 2024 · similarity 0.89)
- Unsupervised Evaluation of Code LLMs with Round-Trip Correctness (ICML 2024 · similarity 0.87)
- NextCoder: Robust Adaptation of Code LMs to Diverse Code Edits (ICML 2025 · similarity 0.87)
- ToolACE: Winning the Points of LLM Function Calling (ICLR 2025 · similarity 0.86)
- CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets (ICLR 2024 · similarity 0.86)
- How efficient is LLM-generated code? A rigorous & high-standard benchmark (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.