AI Paper Atlas

Research map / Large language models / Agents, code and math

Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning

Minju Seo, Jinheon Baek, Seongyun Lee, Sung Ju Hwang

ICLR 2026 · Large language models › Agents, code and math · approach: Software engineering agents · 89 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use full paper instead of only the abstract (ablation)one-shot code generation · Paper2CodeBench ICLR (reference-based)score (1-5): 2.28 -> 3.08Improved
add planning, analysis and staged generation over one-shot full-paper generationPaperCoder · Paper2CodeBench ICLR (reference-based)score (1-5): 3.08 -> 3.68Improved
use PaperCoder instead of ChatDevChatDev · Paper2CodeBench NeurIPS (reference-based)score (1-5): 2.96 -> 3.83Improved
PaperCoder compared with official repositoriesPaperCoder · Paper2CodeBench ICLR (reference-free)score (1-5): 4.84 vs 4.73No clear effect
remove the methodology section from the input paper (ablation)PaperCoder · Paper2CodeBench subsamplereference-based score (1-5): 4.26 -> 3.75Hurt
use PaperCoder instead of IterativeAgentIterativeAgent (o3-mini) · PaperBench Code-Devreplication score (%): 16.4 -> 45.14Improved
use PaperCoder with Claude 3.5 Sonnet instead of IterativeAgentIterativeAgent (Claude 3.5 Sonnet) · PaperBench Code-Devreplication score (%): 27.5 -> 51.14Improved
use a stronger backbone (o3 instead of o3-mini) (ablation)PaperCoder · PaperBench Code-Devreplication score (%): 45.14 -> 60.86Improved
use o1 backbone instead of o3-mini (ablation)PaperCoder · PaperBench Code-Devreplication score (%): 45.14 -> 38.31Hurt
use o3-mini-high rather than GPT-4o as evaluator (ablation)LLM judge · Paper2CodeBenchrank correlation with human: 0.74 -> 0.78Improved
add architecture design module alone in the incremental ablation (ablation)PaperCoder · Paper2CodeBench ICMLscoreHurt
add verification-and-refinement to planning and analysis (ablation)PaperCoder · Paper2CodeBench subsample (30 papers)downstream scoreImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy