Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Scaling Laws for Pre-training Agents and World Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLM-style power-law scaling (optimal model/data size vs compute) emerges for generative world modeling and behavior cloning on human gameplay, with coefficients strongly dependent on tokenizer compression and architecture.
Problem
Scaling in embodied AI (world modeling and behavior cloning) is poorly characterized compared to LLMs; prior work mostly shows 'bigger is better' via downstream performance rather than precise compute-optimal scaling laws on pre-training loss.
Approach
Trains families of GPT-2-style transformers on next-token prediction for world modeling and behavior cloning in the infinite-data regime, fits compute-optimal power laws (frontier and parametric fits), and shows the optimal model/data coefficients depend on tokenizer compression rate and on whether observations are tokenized or encoded as a single continuous embedding.
Key results
- WM-Token-256 yields optimal model and dataset exponents of about 0.49 and 0.51, matching LLM scaling coefficients; WM-Token-540 shifts the model exponent to 0.62.
- BC-Token-540 yields a model exponent of 0.32 (parametric fit) and does not saturate for models above 2M parameters, so frontier fit fails; BC-CNN gives model exponent 0.66 (frontier fit).
- An 894M WM-Token-256 extrapolation run at ~1.58e21 FLOPs agrees well with predicted compute-optimal model and data sizes; pre-training loss correlates ~0.77-0.83 with LPIPS/FVD world-model quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase tokens per image observation from 256 to 540 (ablation) | World modeling with tokenized observations · Bleeding Edge | optimal model-size exponent: 0.49 to 0.62 | Helped |
| Increase tokens per image observation from 256 to 540 (ablation) | World modeling with tokenized observations · Bleeding Edge | optimal dataset-size exponent: 0.51 to 0.37 | Hurt |
| Use sparse action-token supervision instead of dense supervision (ablation) | Character-level language modeling · Shakespeare | optimal model-size exponent: 0.63 to 0.15 | Hurt |
| Super-class the sparse target into two classes (ablation) | Character-level language modeling with sparse final-token loss · Shakespeare | optimal model-size exponent: 0.63 to 0.15 | Hurt |
| Replace tokenized image observations with one continuous CNN embedding per observation (ablation) | Behavior cloning · Bleeding Edge 7 maps | optimal model-size exponent: 0.32 to 0.66 | Helped |
| Replace tokenized image observations with one continuous CNN embedding per observation (ablation) | Behavior cloning · Bleeding Edge 7 maps | optimal dataset-size exponent: 0.68 to 0.34 | Hurt |
| Replace character-level tokenization with the more compressed GPT-2 tokenizer (ablation) | Language modeling · BookCorpus | optimal model-size exponent: 0.66 to 0.44 | Hurt |
| Increase the number of tokens per observation (ablation) | World modeling with RT-1 · RT-1 | optimal model-size exponent | Helped |
| Train an 894M-parameter model beyond the compute range used to fit scaling laws | WM-Token-256 · Bleeding Edge Sky Garden | agreement with predicted compute-optimal loss, model size, and training tokens: 894M; 1.58 × 10^21 FLOPS | Helped |
Surprising findings
- Sharing the same transformer architecture, WM-Token and BC-Token produce opposite skews in optimal model vs data scaling, driven by the dense vs sparse loss and target granularity.
- Tokenizer compression rate changes the optimal model exponent in both world modeling and character-level/GPT-2-tokenized language modeling.
Limitations
- Infinite-data regime only, on a fixed offline dataset; alternative architectures and dataset qualities not studied.
- Pre-training loss is an intermediate proxy; downstream task performance and inference cost are not measured.
- BC-Token scaling could not use the frontier fit method, relying on a parametric fit with strong assumptions.
Experimental setup
Datasets: Bleeding Edge 7 maps dataset (human gameplay) Sky Garden dataset (Bleeding Edge single map) RT-1 robotics dataset Shakespeare character dataset BookCorpus
Metrics: pre-training cross-entropy loss frontier fit coefficients N_optimal and D_optimal exponents parametric fit coefficients FVD LPIPS
Baselines: Chinchilla parametric fit Kaplan-style LLM coefficients VPT-style BC-CNN
Builds on: Kaplan et al. scaling laws Chinchilla compute-optimal scaling Gato VPT
Closest papers by meaning
- Observational Scaling Laws and the Predictability of Langauge Model Performance (NeurIPS 2024 · similarity 0.88)
- Data Scaling Laws in Imitation Learning for Robotic Manipulation (ICLR 2025 · similarity 0.88)
- A Tale of Tails: Model Collapse as a Change of Scaling Laws (ICML 2024 · similarity 0.88)
- A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.87)
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.87)
- GenRL: Multimodal-foundation world models for generalization in embodied agents (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.