Research map / Multi-agent systems and games / LLM agents

CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CrafterDojo, a suite of Crafter foundation models (C-VPT, C-CLIP, C-Steve-1) and data generators (CrafterPlay, CrafterCaption) to serve as a lightweight Minecraft-like testbed.

Problem

Crafter is a lightweight alternative to Minecraft for embodied agent research but lacks foundation models and large-scale behavior and caption datasets needed to develop them.

Approach

Automatically generate behavior demonstrations with a Craftax-trained PPO expert and captions with a rule-based generator, then port VPT, MineCLIP, and Steve-1 recipes to Crafter, adding event-based packed hindsight relabeling and a CVAE text-to-goal prior.

Key results

• C-VPT (large) reaches 61.4% Crafter Score and C-VPT (base) 71.8% return, outperforming prior baselines (e.g., 31.8% score for the strongest prior method).
• C-CLIP reaches 89.8% R@1 on CrafterCaption held-out episodes versus 1.7% for CLIP4Clip.
• Instruction chaining with a heuristic planner raises C-Steve-1 success on T4 from 33% to 80%.

Surprising findings

• C-Steve-1 with a single instruction underperforms on multi-step tasks, motivating planning.
• Removing all noop actions hurts C-VPT, so selective filtering of short noop runs is needed.

Limitations

• Multi-step tasks are weak for single-instruction C-Steve-1 and rely on hand-designed or PPO planners.
• Long-horizon evaluations use a simplified Crafter variant without health decay and mobs for T1.
• Caption and data generation rely on rule-based templates with limited linguistic diversity before paraphrasing.

Experimental setup

Datasets: CrafterPlay CrafterCaption

Benchmarks: Crafter

Metrics: Crafter Score Return Recall@1 Recall@5 Recall@10 Mean Rank success rate

Baselines: Δ-IRIS CLIP4Clip PPO-RNN C-VPT-RL Single Instruction C-Steve-1

Builds on: VPT MineCLIP Steve-1 Craftax LaCLIP

Closest papers by meaning

• ADAM: An Embodied Causal Agent in Open-World Environments (ICLR 2025 · similarity 0.88)
• MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception (CVPR 2024 · similarity 0.88)
• LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence (ICML 2025 · similarity 0.87)
• MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning (NeurIPS 2025 · similarity 0.87)
• Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (NeurIPS 2026 · similarity 0.87)
• SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning (NeurIPS 2026 · similarity 0.87)