Research map / Large language models / Agents, code and math
Adaptive Self-improvement LLM Agentic System for ML Library Development
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an adaptive self-improvement learning algorithm and a multi-agent (proposer/guardian) system that lets LLMs write ML library operators in the STeP streaming dataflow ASPL from self-generated experience.
Problem
Generating high-performance ML library code in architecture-specific programming languages (ASPLs) is hard for LLMs because the task requires complex reasoning and there are very few code examples for esoteric, evolving ASPLs.
Approach
An adaptive self-improvement loop that parallel-samples an LLM agentic system, filters one minimal-length correct answer per AST-isomorphism group, stratifies earned experiences by difficulty (success rate), and adaptively selects hard-to-mixed demonstrations, dropping solved tasks so harder tasks get more test-time compute.
Key results
- The agentic system with self-improvement solves up to 96% of tasks and reaches up to 3.9x improvement over a baseline single LLM (Pass@n).
- Under token-count matching, Claude Sonnet Pass@n goes from 0.77 (single) to 0.96 (agent and self-improved); GPT-4o goes from 0.38 to 0.81.
- Self-improvement reaches Pass@n 0.81 vs 0.62 for SFT-finetuned GPT-4o (from 0.35 starting point).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adaptive self-improvement to the agentic system | Claude Sonnet · STeP benchmark, token-matched | Pass@n: 0.77 -> 0.96 | Helped |
| Add adaptive self-improvement to the agentic system | GPT-4o · STeP benchmark, token-matched | Pass@n: 0.38 -> 0.81 | Helped |
| Apply adaptive self-improvement instead of supervised fine-tuning | GPT-4o · STeP benchmark | Pass@n: 0.62 -> 0.81 | Helped |
| Set adaptive granularity to m=3 instead of m=4 (ablation) | GPT-4o adaptive self-improvement · STeP benchmark | input tokens: 1.07× | Helped |
| Set adaptive granularity to m=3 instead of m=1 (ablation) | GPT-4o adaptive self-improvement · STeP benchmark | performance: 1.5× | Helped |
| Prioritize hard-only demonstrations over demonstrations mixed with easier examples | Adaptive self-improvement · STeP benchmark, learning curve | Pass@n | Helped |
| Use mixed demonstrations to discover new hard-only examples | Adaptive self-improvement · STeP benchmark, DeepSeek-V3 | Pass@n | Mixed |
| Add the proposer-and-guardian agentic organization instead of a single LLM | STeP code generation · STeP benchmark, all tasks | Pass@n: 0.62 -> 0.85 | Helped |
| Add the guardian agent (ablation) | Proposer agent · STeP benchmark, Avg-Reuse | Pass@n: 6/12 -> 11/12 | Helped |
| Add the guardian agent (ablation) | Proposer agent · STeP benchmark, Arith-Once | Pass@1: 0.685 -> 0.663 | Hurt |
| Add structural IR (ablation) | Single LLM coding system · STeP benchmark | Pass@n: 0.46 -> 0.62 | Helped |
| Add structural IR (ablation) | Single LLM coding system · STeP benchmark | success diversity: 0.32 -> 0.27 | Hurt |
Surprising findings
- Structural IR improves Pass@n by increasing sample diversity, though it can hurt success diversity.
- The guardian agent fixes affine type errors but can reduce Pass@1 on some tasks by corrupting correct proposer outputs.
Limitations
- Verification is functional only; simulator restricts dynamism to what ML operators need.
- Adaptive selection is limited by context length, requiring better stratification for larger task sets.
- Evaluated on a single ASPL (STeP) with 26 tasks.
Experimental setup
Datasets: custom STeP benchmark of 26 ML operator tasks AIME-2024
Benchmarks: STeP ML library benchmark (26 tasks) AIME-2024
Metrics: pass@k Pass@1 Pass@n input tokens semantic diversity
Baselines: single LLM single LLM without structural IR OpenAI-o1 supervised finetuning (GPT-4o)
Builds on: pass@k curriculum learning STeP
Closest papers by meaning
- CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments (ICML 2026 · similarity 0.86)
- Commit0: Library Generation from Scratch (ICLR 2025 · similarity 0.86)
- Huxley-G\"odel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine (ICLR 2026 · similarity 0.86)
- Meta$^n$: Recursive Self-Improvement through Emergent Depth (NeurIPS 2026 · similarity 0.86)
- CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets (ICLR 2024 · similarity 0.86)
- Steering Large Language Models between Code Execution and Textual Reasoning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.