Research map / Architectures and efficiency / Neural architecture search

Simple Baselines are Competitive with Code Evolution

NeurIPS 2026 · Architectures and efficiency › Neural architecture search · approach: Bi-level differentiable NAS · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes two simple code-evolution baselines (IID random sampling and sequential conditioned sampling) and shows they match or exceed ShinkaEvolve, AlphaEvolve-style bounds search, ADAS, and AIDE across math bounds, agentic scaffold design, and MLE-bench Kaggle tasks.

Problem

Code evolution pipelines for LLM-driven program search are complex and are rarely compared against simple baselines under matched budgets, prompts, and models, so reported gains may not reflect the search method.

Approach

Two minimal baselines: IID random sampling of programs from an LLM and sequential conditioned sampling that conditions new programs on random successful programs from the previous generation, with no explicit fitness-based selection. They are compared under equal API budgets, evaluations, or wall-clock time across three domains.

Key results

• On nine AlphaEvolve math bounds under a $20 budget, SCS matches or exceeds ShinkaEvolve on 6/9 problems and IID RS on 4/9.
• On AIME scaffold design, all automated search methods including IID RS drop over 10% from validation to test, while majority vote baselines degrade about 5%.
• On the 10-competition MLE-bench subset, SCS matches or exceeds AIDE on 6/10 competitions and IID RS on 3/10.

Surprising findings

• For circle packing, changing the problem formulation (uncertainty inequality) improved bounds more than changing the search pipeline, and all methods reached the same bound.
• Hand-designed majority vote scaffolds outperformed all automated scaffold search methods on AIME 2025 test accuracy.

Limitations

• Single ShinkaEvolve run per problem due to cost, so comparisons have high uncertainty.
• Scaffold evaluation on small validation sets (~100 questions) leads to high stochasticity; recommendations are not fully resolved for evolution.
• Many code evolution pipelines lack standardization, making fair comparisons costly and sometimes not possible (e.g., OpenEvolve failures).

Experimental setup

Datasets: AlphaEvolve math problems (9) AIME 2024 AIME 2025 MLE-bench 10-competition subset

Benchmarks: AlphaEvolve math bounds (9 problems) AIME 2024/2025 MLE-bench (10 Kaggle competitions)

Metrics: best bound found accuracy probability of improvement probability of matching or exceeding probability of dominance medal/median rank

Baselines: ShinkaEvolve AlphaEvolve OpenEvolve ADAS AIDE majority vote@k

Builds on: AlphaEvolve ShinkaEvolve ADAS AIDE FunSearch

Closest papers by meaning

• From I/O to Code with Discovery Agent (NeurIPS 2026 · similarity 0.90)
• Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI (ICML 2025 · similarity 0.89)
• CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments (ICML 2026 · similarity 0.88)
• Planning in Natural Language Improves LLM Search for Code Generation (ICLR 2025 · similarity 0.88)
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks (NeurIPS 2026 · similarity 0.88)
• Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs (NeurIPS 2025 · similarity 0.87)