Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Quantifying Generalisation in Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Labyrinth, a discrete, fully observable gymnasium maze environment with controllable train/eval/test splits, key-and-door and ice-floor variants, and benchmarks six imitation learning methods, which generalise poorly to unseen structures.
Problem
Imitation learning benchmarks (CartPole, MountainCar, MuJoCo, Atari) often lack meaningful separation between training and evaluation conditions and do not expose optimal actions per state, which limits assessment of generalisation.
Approach
A configurable labyrinth environment where structure, start and goal positions can be varied to create verifiably distinct train/eval/test splits, with a discrete graph allowing computation of optimal actions in every state, plus partial observability, key-and-door, and ice-floor variants.
Key results
- On the 5x5 navigation task, IUPE reaches 75% success on training but only 21% on validation and 5% on test; all methods reach 0% when both start and goal change.
- BC with a ResNet-18 encoder achieves 100% training SR, 56% validation and 53% test SR, improving over the default CNN BC baseline.
- On the 3x3 and 4x4 settings, IUPE is best on 4x4 with 68% test SR, while SQIL underperforms on most splits.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase BC training from 1,000 to 10,000 epochs (ablation) | BC with Atari DQN CNN · 5×5 navigation | success ratio: 6% -> 41% validation; 2% -> 34% test | Helped |
| Replace the Atari DQN CNN encoder with ResNet-18 (ablation) | BC · 5×5 navigation | success ratio: 6% -> 56% validation; 2% -> 53% test | Helped |
| Use a larger labyrinth setting, from 3×3 to 5×5 | BC · navigation test | success ratio: 65.62% -> 2% | Hurt |
| Use IUPE instead of BC | 5×5 navigation · validation | success ratio: 6% -> 21% | Helped |
| Use IUPE instead of BC | 5×5 navigation · test | success ratio: 2% -> 5% | Helped |
Surprising findings
- Across MuJoCo-style and classic control benchmarks, replaying a single expert action sequence or nearest-state actions reaches near-expert reward, suggesting these benchmarks are lenient about generalisation.
- Partially observable labyrinths are hypothesised, not shown, to be easier for imitation learning than full observation.
Limitations
- Only discrete actions are supported, so continuous-action imitation methods cannot be evaluated.
- Custom tile behaviours require changing the environment source code.
- Benchmark results are reported for the navigation setting only in the main text; other settings appear only in the supplement.
Experimental setup
Datasets: Labyrinth 3x3, 4x4, 5x5 datasets (shortest paths, biased setting)
Benchmarks: Labyrinth navigation 5x5 train/validation/test splits Labyrinth 3x3 and 4x4 splits Labyrinth s0 and s0-and-g splits
Metrics: average episodic reward success ratio
Baselines: BC DAgger GAIL BCO SQIL IUPE
Builds on: gymnasium IL-Datasets Deep Q-Network ResNet
Closest papers by meaning
- Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames (NeurIPS 2025 · similarity 0.88)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.88)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.88)
- Diffusing States and Matching Scores: A New Framework for Imitation Learning (ICLR 2025 · similarity 0.88)
- Data Scaling Laws in Imitation Learning for Robotic Manipulation (ICLR 2025 · similarity 0.87)
- Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.