Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Bootstrapping Cognitive Agents with a Large Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Bootstraps a SOAR-style cognitive agent's production rules by querying GPT-4 for actions and generalized rules, then reuses learned productions to cut LLM token use in AI2-THOR kitchen tasks.
Problem
Cognitive architectures require heavy manual construction of production rules, while LLMs hold noisy common-sense knowledge that is hard to update; the paper aims to combine them for embodied kitchen tasks.
Approach
An LLM (GPT-4) is queried only when no production applies; its suggested action is converted into generalized, parameterized production rules (English description then Python code), refined by a critic LLM, and reinforced by a utility update so the agent reuses them without further LLM calls.
Key results
- Bootstrapped agent succeeds on 15/15 for find, slice and clear-countertops tasks, matching the action-only baseline's 14/15 and 15/15 and 15/15.
- Bootstrapped agent uses 916.87 tokens on find versus 54754.20 for action-only, and 0 tokens on slice and clear tasks after bootstrapping.
- No significant step-count difference for find (p=0.446) or slice (p=0.347), but the bootstrapped agent takes significantly more steps on clearing countertops (p=0.001).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · find a/an <object> | Success: 14 / 15 -> 15 / 15 | Helped |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · slice a/an <object> | Steps: 28.20 -> 29.13 | No clear effect |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · find a/an <object> | Steps: 15.67 -> 15.80 | No clear effect |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · clear the countertops | Steps: 5.13 -> 7.47 | Hurt |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · find a/an <object> | Tokens: 54754.20 -> 916.87 | Helped |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · slice a/an <object> | Tokens: 102806.60 -> 0.00 | Helped |
| Use the bootstrapped cognitive agent instead of action-only LLM queries | GPT4-0613 kitchen agent · clear the countertops | Tokens: 18924.87 -> 0.00 | Helped |
Surprising findings
- The bootstrapped agent completes find tasks for objects not in the training scene with very limited LLM queries, but the action-only baseline wrongly treated mug as cup and failed one instance.
- The learned clearing production causes the bootstrapped agent to use more steps by placing each object in a separate empty receptacle.
Limitations
- Operates only at high-level decision making and relies on well-defined low-level navigation and manipulation interfaces; sim-to-real gap expected.
- Requires skills whose decision process can be articulated in language; hard for non-verbal skills.
- Results are not fully reproducible due to GPT-4 and simulator nondeterminism, and the agent is evaluated on a small set of 15 instances per task.
Experimental setup
Datasets: AI2-THOR kitchen environments
Benchmarks: AI2-THOR kitchen tasks (find, slice, clear countertops)
Metrics: success rate steps tokens used paired t-test p-values
Baselines: action-only LLM agent
Builds on: SOAR ACT-R Voyager Chain-of-thought prompting
Closest papers by meaning
- Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.87)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.87)
- Turning large language models into cognitive models (ICLR 2024 · similarity 0.87)
- Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks (ICLR 2024 · similarity 0.87)
- Embodied CoT Distillation From LLM To Off-the-shelf Agents (ICML 2024 · similarity 0.87)
- ModularAgent: A Task-Aware Modular Framework for Joint Optimization of Multimodal Large Language Models and World Models (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.