Research map / Large language models / Agents, code and math
Position: Foundation Agents as the Paradigm Shift for Decision Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that foundation agents will be a paradigm shift for decision making, outlining their characteristics, a roadmap (large interactive data, self-supervised pretraining, LLM alignment), research questions, and a small DMC/Atari pretraining case study.
Problem
Conventional decision-making agents (RL, IL, planning) suffer from low sample efficiency and poor generalization, and training agents from scratch per task does not scale like foundation models in language and vision.
Approach
Frames foundation agents as generally capable decision-making agents built via large interactive data, self-supervised sequence-model pretraining on trajectories, and alignment with LLMs for world knowledge and reasoning, and outlines the open challenges this raises.
Key results
- Case study: an 25M-parameter GPT-2-based agent jointly pretrained on 5 DMC tasks and 5 Atari games via next-action prediction is fine-tuned on an unseen DMC walker-walk task and Pong, shown only qualitatively.
- Reported pretraining compute for the case study is 8 hours on 4 RTX4090 GPUs.
- Table of prior generalist agents shows Gato at 1.18B parameters trained for 4 days on 256 TPUv3.
Limitations
- Case study is a single qualitative visualization with no quantitative baseline comparison.
- Many proposed directions (LLM alignment, theoretical guarantees, open-ended tasks) are not empirically validated.
- Video generation models as simulators are acknowledged as insufficient for many decision tasks.
Experimental setup
Datasets: DeepMind Control Suite Atari
Benchmarks: DeepMind Control Suite walker walk Atari Pong
Metrics: fine-tuning return (visualized)
Baselines: Gato Multi-Game Decision Transformer DualMind Interactive Agent Foundation Model
Builds on: Decision Transformer Gato GPT-2 SMART
Closest papers by meaning
- ExpeL: LLM Agents Are Experiential Learners (AAAI 2024 · similarity 0.87)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.86)
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.86)
- Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.86)
- Can large language models explore in-context? (NeurIPS 2024 · similarity 0.85)
- Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.