Research map / Multi-agent systems and games / Human-AI and goal-conditioned
BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Tasks
Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander V Schulhoff, Brandon Houghton, Rohin Shah
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| human agents instead of best algorithmic agent GoUp | BASALT agent · BASALT evaluation | normalized TrueSkill: 0.73 -> 2.43 | Improved |
| GoUp (hand-decomposed tasks plus vision) instead of UniTeam | BASALT agent · BASALT evaluation | normalized TrueSkill: 0.13 -> 0.73 | Improved |
| UniTeam (BC plus VPT embedding search) instead of BC-Baseline | BASALT agent · BASALT evaluation | normalized TrueSkill: -0.32 -> 0.13 | Improved |
| BC-Baseline instead of Random agent | BASALT agent · BASALT evaluation | normalized TrueSkill: -1.35 -> -0.32 | Improved |
| human players in comparison vs algorithmic agents | judge justifications · BASALT evaluation | positive sentiment (%): 63 -> 92 | Improved |
| no machine learning agent beats humans on any extra factor | ML agents · BASALT evaluation | human-judged factors | Hurt |
| task BuildVillageHouse vs CreateVillageAnimalPen | human demonstrations · Demonstrations Dataset | mean video length (minutes): 1.98 -> 6.52 | Hurt |
| FindCave vs CreateVillageAnimalPen | judge justifications · Evaluation Dataset | positive sentiment (%): 57 -> 79 | Improved |
| MakeWaterfall vs CreateVillageAnimalPen | judge justifications · Evaluation Dataset | positive sentiment (%): 57 -> 76 | Improved |
| AnimalPen vs BuildVillageHouse sentiment distribution | judge justifications · Evaluation Dataset | sentiment distribution: 6.35 | No clear effect |
| FindCave vs MakeWaterfall sentiment distribution | judge justifications · Evaluation Dataset | sentiment distribution: 2.694 | No clear effect |
| FindCave and MakeWaterfall vs CreateVillageAnimalPen response length | judge justifications · Evaluation Dataset | words per response: F=3.72 | Improved |
Closest papers by meaning
- MineObserver 2.0: A Deep Learning & In-Game Framework for Assessing Natural Language Descriptions of Minecraft Imagery (AAAI 2024 · similarity 0.86)
- Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft (CVPR 2024 · similarity 0.84)
- MCU: An Evaluation Framework for Open-Ended Game Agents (ICML 2025 · similarity 0.82)
- MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memory (ICLR 2025 · similarity 0.82)
- CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter (NeurIPS 2026 · similarity 0.82)
- AssistanceZero: Scalably Solving Assistance Games (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
