Research map / Multi-agent systems and games / LLM agents

Refactoring Code Through Library Design

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies which metrics make a good code library refactoring (finding MDL best matches human preference), then proposes Librarian, a cluster-then-sample-and-rerank method, and the MiniCode benchmark for multi-file library refactoring.

Problem

Code agents solve isolated programming problems but do not consolidate specialized code into maintainable, reusable libraries; it is unclear which metric should judge refactoring quality and how to generate reusable libraries from many files.

Approach

Clusters files by LLM-generated summaries, samples K candidate library refactorings per cluster, filters by test pass rate, and reranks by minimum description length (MDL) under a small reference LM; MDL is chosen as the metric after comparing against tokens, cyclomatic complexity, and maintainability index.

Key results

• On 10 CodeContests tasks (K=8, S=3), Librarian reaches MDL ratio 0.53 and token ratio 0.66 while pass rate rises to 90.67%.
• On Logo and Date holdout program synthesis, Librarian with gpt-3.5-turbo improves over REGAL: 69.9% vs 49.3% on Logo and 94.7% vs 90.2% on Date.
• On real Transformers files, Librarian lowers MDL to 67.2% of original while passing all integration tests; a human-authored reference reached 62%.

Surprising findings

• Neural language model MDL gives a stronger signal of refactoring quality than classic software metrics such as maintainability index and cyclomatic complexity, matching human preferences.
• Nearly 40% of learned library functions are used only once, which the authors argue is a feature because MDL favors natural decompositions.

Limitations

• Logo and Date are synthetic toy problems and CodeContests is not naturalistic.
• Compression by tokens or MDL may not always correlate with reuse.
• Refactoring at scale is expensive, about 30 minutes per refactoring.

Experimental setup

Datasets: CodeContests Logo Date Huggingface Transformers modeling files Huggingface Diffusers UNet and scheduler files

Benchmarks: MiniCode CodeContests Logo Date Transformers Diffusers

Metrics: MDL ratio token ratio pass rate solve rate calls per library function Bradley-Terry preference

Baselines: REGAL Codex-mini Claude Sonnet 4 Claude Sonnet 3.7 Tokens metric Maintainability Index Cyclomatic Complexity

Builds on: REGAL DreamCoder Commit-0

Closest papers by meaning

• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks (NeurIPS 2026 · similarity 0.85)
• MaintainCoder: Maintainable Code Generation Under Dynamic Requirements (NeurIPS 2025 · similarity 0.84)
• ReGAL: Refactoring Programs to Discover Generalizable Abstractions (ICML 2024 · similarity 0.84)
• Build-and-Find: An Effort-Aware Protocol for Evaluating Agent-Managed Codebases (NeurIPS 2026 · similarity 0.83)
• SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades (NeurIPS 2026 · similarity 0.83)
• Code Repair with LLMs gives an Exploration-Exploitation Tradeoff (NeurIPS 2024 · similarity 0.83)