Research map / Multi-agent systems and games / Human-AI and goal-conditioned
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fine-tuned MineCLIP dense reward instead of sparse-only reward | PPO agent · Programmatic tasks (Hunt Cow) | success rate (%): 0.3 -> 83.5 | Helped |
| use MineCLIP (Attn) reward instead of manually engineered dense reward | PPO agent · Programmatic tasks | success rate (%): Hunt Cow 48.3 -> 83.5; Shear Sheep 52.3 -> 12.1 | Mixed |
| use MineCLIP fine-tuned on Minecraft YouTube videos instead of original OpenAI CLIP (ablation) | reward model · Programmatic tasks (Hunt Cow) | success rate (%): 0.0 -> 83.5 | Helped |
| replace transformer attention aggregator with average pooling (ablation) | MineCLIP · Hunt Cow | success rate (%): 83.5 -> 0.0 | Hurt |
| replace transformer attention aggregator with average pooling (ablation) | MineCLIP · Find Nether Portal | success rate (%): 37.4 -> 89.8 | Helped |
| replace transformer attention aggregator with average pooling (ablation) | MineCLIP · Combat Zombie | success rate (%): 47.3 -> 72.3 | Helped |
| use MineCLIP video encoder instead of OpenAI CLIP encoder for visual generalization (ablation) | policy encoder · unseen weather/lighting/terrain | relative performance drop: Hunt Cow -32.9% vs -77.0% | Helped |
| train one multi-task agent on all 12 tasks instead of per-group agents (ablation) | PPO agent · 12 MineDojo tasks | success rate (%): Milk Cow 64.5 -> 91.5; Hunt Cow 83.5 -> 46.8 | Mixed |
| pre-train multi-task agent then RL finetune instead of RL from scratch | PPO agent · Hunt Pig | success rate (%): 0.0 -> 46.0 | Helped |
| pre-train multi-task agent then RL finetune instead of RL from scratch | PPO agent · Harvest Spider String | success rate (%): 12.5 -> 36.5 | Helped |
| use MineCLIP instead of OpenAI CLIP as automatic evaluator | evaluation metric · Find Nether Portal | F1 vs human (%): 48.7 -> 98.7 | Helped |
Closest papers by meaning
- ADAM: An Embodied Causal Agent in Open-World Environments (ICLR 2025 · similarity 0.92)
- CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter (NeurIPS 2026 · similarity 0.91)
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks (NeurIPS 2024 · similarity 0.89)
- LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence (ICML 2025 · similarity 0.89)
- OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents (NeurIPS 2024 · similarity 0.89)
- MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.