Research map / Multi-agent systems and games / Human-AI and goal-conditioned
WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WorldAPIs, a thought experiment that iteratively induces an open primitive-action (API) space by few-shot prompting GPT-4 to write Pythonic agent policies for wikiHow tutorials, yielding 300+ APIs from under 0.5% of tutorials.
Problem
How many primitive actions (APIs) does a versatile embodied agent need to carry out diverse real-world daily tasks, and what should they look like? Existing embodied simulators use small, hand-crafted, closed action spaces.
Approach
Top-down induction: starting from wikiHow procedural instructions, few-shot prompt an LLM to write Pythonic agent programs that reuse seed APIs and hallucinate new ones when needed, bootstrapping the API pool from its own verified outputs, then measure overlap with existing simulator actions.
Key results
- Induction on under 0.5% of wikiHow tutorials yields an API pool that saturates at roughly 300+ frequently-evoked APIs (plateau after ~600 tutorials).
- Adding API use-case demonstrations cut new API creation from 2.88 to 1.24 per tutorial; adding descriptions raised faithfulness to 84.0 and best ranking (1.439).
- Only 9 of the top 50 induced APIs are supported by existing embodied simulators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add API use-case demonstrations (ablation) | WorldAPIs Base · wikiHow, first 50 tutorials | Redundancy score: 46.50 -> 43.44 | Helped |
| add API use-case demonstrations (ablation) | WorldAPIs Base · wikiHow, first 50 tutorials | New APIs per tutorial: 2.88 -> 1.24 | Hurt |
| add API use-case demonstrations (ablation) | WorldAPIs Base · wikiHow, first 50 tutorials | Faithfulness score: 82.0 -> 81.0 | No clear effect |
| add instruction descriptions to the use-case variant (ablation) | WorldAPIs Base + Use Case · wikiHow, first 50 tutorials | Faithfulness score: 81.0 -> 84.0 | Helped |
| add instruction descriptions to the use-case variant (ablation) | WorldAPIs Base + Use Case · wikiHow, first 50 tutorials | Redundancy score: 43.44 -> 47.46 | Hurt |
| add instruction descriptions to the use-case variant (ablation) | WorldAPIs Base + Use Case · wikiHow, first 50 tutorials | Faithfulness ranking: 1.732 -> 1.439 | Helped |
Surprising findings
- Existing embodied simulators cover only 9 of the top 50 frequent induced APIs, indicating a large action-space gap.
- Adding use-case demonstrations reduced API creation but slightly lowered the non-complex redundancy score compared with the Full variant in some metrics.
Limitations
- Induced action space is noisy, with a relatively high portion of redundant APIs.
- No execution-based evaluation; faithfulness is approximated by human annotation without a simulator.
Experimental setup
Datasets: wikiHow Home & Garden tutorials
Benchmarks: wikiHow Home & Garden (1000 sampled tutorials)
Metrics: redundancy score (0-0.5-1) faithfulness score (0-0.5-1) faithfulness pairwise ranking API pool size new API induction ratio API frequency
Baselines: Full (no use-case demonstrations) Base + Use Case existing embodied simulators (ALFRED, VirtualHome, TEACh, BEHAVIOR, etc.)
Builds on: ProgPrompt-style LLM code planning wikiHow procedural instruction datasets AI2-THOR
Closest papers by meaning
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.86)
- What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces? (ICML 2026 · similarity 0.85)
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.85)
- UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.84)
- Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.84)
- World Model Implanting for Test-time Adaptation of Embodied Agents (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.