Research map / Large language models / Agents, code and math
Fault-Aware Neural Code Rankers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add fault-aware CodeRanker reranking of 100 samples | Codex · APPS validation | pass@1 (%): 26 -> 39.6 | Helped |
| add CodeRanker reranking | GPT-Neo 125M · APPS validation | pass@1 (%): 1.4 -> 6.5 | Helped |
| add CodeRanker reranking | Codex · APPS test | pass@1 (%): 3.8 -> 4.5 | Mixed |
| add CodeRanker reranking trained on APPS, zero-shot | Codex · HumanEval | pass@1 (%): 26.3 -> 32.3 | Helped |
| add CodeRanker reranking trained on APPS, zero-shot | Codex · MBPP | pass@1 (%): 36.4 -> 41.8 | Helped |
| add CodeRanker reranking | GPT-J · MBPP | pass@5 (%): 28.9 -> 28.2 | Hurt |
| lower sampling temperature to 0.2 with ranker (ablation) | Codex · HumanEval | ranked pass@1 (%): 32.3 -> 42.7 | Helped |
| add CodeRanker under low temperature 0.2 | Codex · MBPP | pass@1 (%): 47.2 -> 46.8 | Hurt |
| use ternary/fine-grained fault-aware ranker targets instead of binary (ablation) | CodeRanker · APPS validation | ranked pass@1 | Helped |
| mix ranker datasets from all four code generators (mixed-large) (ablation) | CodeRanker · APPS validation | ranked pass@1 | Helped |
| train ranker on data from a different-size code generator (ablation) | CodeRanker · APPS validation | ranked pass@1 | Hurt |
Closest papers by meaning
- CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.86)
- CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking (ICLR 2025 · similarity 0.85)
- rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.84)
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.84)
- EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning (ICML 2025 · similarity 0.84)
- AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.