AI Paper Atlas

Research map / Reinforcement learning / Imitation learning

The MAGICAL Benchmark for Robust Imitation

Sam Toyer, Rohin Shah, Andrew Critch, Stuart J. Russell

NeurIPS 2020 · Reinforcement learning › Imitation learning · approach: Offline imitation from imperfect data · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Test on Layout variant instead of Demo variant (object positions changed)BC · MAGICALmean test score: 0.64±0.29 -> 0.14±0.16Hurt
Test on Layout variant instead of Demo variantGAIL · MAGICALmean test score: 0.72±0.35 -> 0.22±0.23Hurt
Test on Colour variant instead of Demo variantGAIL · MAGICALmean test score: 0.72±0.35 -> 0.27±0.24Hurt
Remove all augmentations (ablation)BC · MAGICAL Jitter variantmean test score: 0.56±0.27 -> 0.37±0.30Hurt
Remove all augmentations (ablation)BC · MAGICAL Demo variantmean test score: 0.64±0.29 -> 0.55±0.37Hurt
Remove all discriminator augmentations (ablation)GAIL · MAGICAL Demo variantmean test score: 0.72±0.35 -> 0.44±0.42Hurt
Use allocentric instead of egocentric perspective (ablation)BC · MAGICAL Layout variantmean test score: 0.14±0.16 -> 0.04±0.04Hurt
Use allocentric instead of egocentric perspective (ablation)GAIL · MAGICAL Colour variantmean test score: 0.27±0.24 -> 0.39±0.36Improved
Use allocentric instead of egocentric perspective (ablation)BC · MAGICAL Colour variantmean test score: 0.39±0.30 -> 0.42±0.32Improved
Switch GAIL discriminator loss to WGAIL-GP (ablation)GAIL · MAGICAL Demo variantmean test score: 0.72±0.35 -> 0.42±0.38Hurt
Switch GAIL discriminator loss to WGAIL-GP (ablation)GAIL · MAGICAL Colour variantmean test score: 0.27±0.24 -> 0.10±0.11Hurt
Train multi-task instead of single-task (ablation)BC · MAGICAL Demo variantmean test score: 0.64±0.29 -> 0.59±0.33No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy