Research map / Multi-agent systems and games / LLM agents

Self-Programmed Execution for Language-Model Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces self-programmed execution (SPE), where the model completion is itself the orchestrator program, formalized via agentic machines and realized in Spell, a Lisp-based language; frontier models use it on coding and agentic benchmarks without training.

Problem

Existing LM agents rely on a fixed harness orchestration policy governing state transitions between turns; the paper asks whether orchestration can be fully delegated to model-written programs.

Approach

The model completion is evaluated as a program that performs all turn-to-turn orchestration (via recursive self-calls), with no outer agent loop; formalized as an SPE state of an agentic machine, and made practical by Spell, a Lisp dialect with quine forms and a trailing-expression pattern that prevents effect replay.

Key results

• GPT-5.4 in Spell resolved 40/80 Terminal-Bench tasks at high effort vs 43/80 for Codex CLI at roughly 2x the cost.
• On SWE-bench Lite, Spell with GPT-5.4 medium effort resolved 171/300 vs 172/300 for Codex CLI at lower cost.
• On AppWorld Spell scored 42.1% vs 63.2% for Codex CLI, and on LongBench v2 61.0% vs 67.5%.

Surprising findings

• Current models mostly use Spell for context pruning and programmatic tool calling rather than the more flexible orchestration policies they are capable of writing; in orchestration games GPT-5.4 wrote nontrivial programs but did not usually choose to.
• No Spell agent successfully used multi-agent delegation, and instructing agents to attempt it was ineffective.

Limitations

• Smaller models (Qwen3.6 Plus) frequently produce fatal invalid Spell programs; only GPT-5.4 had zero fatal errors.
• Spell lags Codex CLI on AppWorld and LongBench v2 and on high-effort SWE-bench Lite; causes unclear.
• No model trained for SPE or Spell; results rely on in-context learning of the language.

Experimental setup

Datasets: Terminal-Bench 1.1 SWE-bench Lite LongBench v2 AppWorld

Benchmarks: Terminal-Bench 1.1 SWE-bench Lite LongBench v2 AppWorld

Metrics: success rate (resolved tasks) fatal error count total cost token count

Baselines: Codex CLI Claude Code ReAct

Builds on: ReAct Recursive language models CodeAct Clojure CEK machine

Closest papers by meaning

• AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration (ICML 2026 · similarity 0.88)
• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.87)
• RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code (ICLR 2025 · similarity 0.86)
• Self-Challenging Language Model Agents (NeurIPS 2025 · similarity 0.86)
• From an LLM Swarm to a PDDL-empowered Hive: Planning Self-executed Instructions in a Multi-modal Jungle (ICLR 2025 · similarity 0.86)
• Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages (NeurIPS 2026 · similarity 0.86)