Research map / Large language models / Language, safety and interpretability
SWINGARENA: Adversarial Programming Arena for Long-context GitHub Issue Solving
Wendong XU, Jing Xiong, Chenyang Zhao, Qiujiang Chen, Haoran Wang, Hui Shen, Zhongwei Wan, Jianbo Dai, Taiqiang Wu, He Xiao, Chaofan Tao, Zhuoqing Mao and 7 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena C++ | win rate: 0.77 -> 0.84 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena C++ | Best@3: 0.38 -> 0.42 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Python | win rate: 0.71 -> 0.84 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Rust | win rate: 0.72 -> 0.75 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Rust | Best@3: 0.49 -> 0.58 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Go | win rate: 0.71 -> 0.80 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Go | Best@3: 0.37 -> 0.45 | Improved |
| add RACG to the submitter (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena Python | Best@3: 0.44 -> 0.46 | Improved |
| use Top-20 related retrieval instead of BM25 only (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena ablation subset | win rate: 0.62 -> 0.73 | Improved |
| raise retrieval from Top-2 to Top-20 related files (ablation) | Qwen2.5-Coder-7B-Instruct submitter · SwingArena ablation subset | Best@3: 0.42 -> 0.43 | Improved |
| use class-level chunk retrieval instead of BM25 (ablation) | file retrieval · File hit rate Top-10 | hit rate: 20.7% -> 48.7% | Improved |
| use function-level chunks instead of BM25 (ablation) | file retrieval · File hit rate Top-2 | hit rate: 0.182 -> 0.277 | Improved |
Closest papers by meaning
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.88)
- Copilot Arena: A Platform for Code LLM Evaluation in the Wild (ICML 2025 · similarity 0.88)
- EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations (NeurIPS 2024 · similarity 0.87)
- EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning (ICML 2025 · similarity 0.87)
- Rethinking Verification for LLM Code Generation: From Generation to Testing (NeurIPS 2025 · similarity 0.87)
- ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
