AI Paper Atlas

Research map / Multi-agent systems and games / Human-AI and goal-conditioned

BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Tasks

Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander V Schulhoff, Brandon Houghton, Rohin Shah

NeurIPS 2023 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Boundedly rational agent modeling · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
human agents instead of best algorithmic agent GoUpBASALT agent · BASALT evaluationnormalized TrueSkill: 0.73 -> 2.43Improved
GoUp (hand-decomposed tasks plus vision) instead of UniTeamBASALT agent · BASALT evaluationnormalized TrueSkill: 0.13 -> 0.73Improved
UniTeam (BC plus VPT embedding search) instead of BC-BaselineBASALT agent · BASALT evaluationnormalized TrueSkill: -0.32 -> 0.13Improved
BC-Baseline instead of Random agentBASALT agent · BASALT evaluationnormalized TrueSkill: -1.35 -> -0.32Improved
human players in comparison vs algorithmic agentsjudge justifications · BASALT evaluationpositive sentiment (%): 63 -> 92Improved
no machine learning agent beats humans on any extra factorML agents · BASALT evaluationhuman-judged factorsHurt
task BuildVillageHouse vs CreateVillageAnimalPenhuman demonstrations · Demonstrations Datasetmean video length (minutes): 1.98 -> 6.52Hurt
FindCave vs CreateVillageAnimalPenjudge justifications · Evaluation Datasetpositive sentiment (%): 57 -> 79Improved
MakeWaterfall vs CreateVillageAnimalPenjudge justifications · Evaluation Datasetpositive sentiment (%): 57 -> 76Improved
AnimalPen vs BuildVillageHouse sentiment distributionjudge justifications · Evaluation Datasetsentiment distribution: 6.35No clear effect
FindCave vs MakeWaterfall sentiment distributionjudge justifications · Evaluation Datasetsentiment distribution: 2.694No clear effect
FindCave and MakeWaterfall vs CreateVillageAnimalPen response lengthjudge justifications · Evaluation Datasetwords per response: F=3.72Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy