Research map / Large language models / Agents, code and math
CrossPL: Systematic Evaluation of Large Language Models for Cross Programming Language Interoperating Code Generation
zhanhang xiong, Dongxia Wang, Yuekang Li, Xinyuan An, Wenhai Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable think mode in Qwen3 models (ablation) | Qwen3 series · CrossPL-FFI | Pass@1 | Improved |
| enable think mode in Qwen3 models (ablation) | Qwen3 series · CrossPL-IPC | Pass@1 | No clear effect |
| use a strong Extractor instead of Qwen3-4B (ablation) | CrossPL-IPC construction pipeline · CrossPL-IPC construction | valid samples: 1,982 to 1,723 | Improved |
| use a strong Instructor instead of Qwen3-4B (ablation) | CrossPL-IPC construction pipeline · CrossPL-IPC construction | valid samples: 1,982 to 1,890 | Improved |
| use a strong Extractor instead of Qwen3-4B (ablation) | CrossPL-FFI construction pipeline · CrossPL-FFI construction | valid samples: 522 to 65 | Improved |
| increase samples from Pass@1 to Pass@5 | GPT-4o · CrossPL-FFI | pass rate: 19.54% to 26.46% | Improved |
| vary sampling temperature and top-p (ablation) | Locally deployable models · CrossPL-IPC | Pass@1 | No clear effect |
| vary sampling temperature and top-p (ablation) | Locally deployable models · CrossPL-FFI | Pass@1 | No clear effect |
| use newer (2024-2025) samples versus older ones | LLMs evaluated greedy · CrossPL-IPC temporal split | Pass@1 | No clear effect |
Closest papers by meaning
- EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code (NeurIPS 2025 · similarity 0.88)
- ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environments (ICLR 2025 · similarity 0.88)
- McEval: Massively Multilingual Code Evaluation (ICLR 2025 · similarity 0.88)
- How efficient is LLM-generated code? A rigorous & high-standard benchmark (ICLR 2025 · similarity 0.88)
- BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.87)
- CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
