Research map / Reinforcement learning / Imitation learning
Memory-Consistent Neural Networks for Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes memory-consistent neural networks (MCNNs), a semi-parametric behavior cloning model class whose outputs are hard-constrained to permissible regions around prototype memories, and shows gains across MLP, BeT, and diffusion backbones on Adroit, CARLA, and Franka Kitchen.
Problem
Behavior cloning policies make small errors away from expert training states that compound over rollouts, causing task failure, especially with few demonstrations.
Approach
Combine a nearest-memory lookup (memories selected by neural gas from the demonstrations) with a DNN whose output is bounded by tanh and mixed in with weight that vanishes at memories, so the policy exactly matches expert actions at memories and stays within bounded permissible regions elsewhere; a sub-optimality bound follows from the bounded width.
Key results
- MCNN+MLP reaches 3285 on pen-human-v1 vs 1845 for MLP-BC and 2586 for IBC, and 4051 on pen-expert-v1 with tuned hyperparameters.
- On Franka Kitchen, MCNN+Diff reaches 0.08 success on p5 versus 0.02 for Diff-BC, with 1.0 on p1 to p4.
- MCNN+MLP outperforms the nearest baseline by 27% on the CARLA lane task and by over 100% in aggregate on expert Adroit tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MCNN constraints to an MLP behavior-cloning policy | MLP-BC · pen-human-v1 | return: 1845 { $\pm$ 213} -> 3285 { $\pm$ 209} | Helped |
| Add MCNN constraints to an MLP behavior-cloning policy | MLP-BC · hammer-human-v1 | return: -11 { $\pm$ 118} -> 262 { $\pm$ 107} | Helped |
| Add MCNN constraints to a BeT behavior-cloning policy | BeT-BC · relocate-human-v1 | return: -2.7 { $\pm$ 1.6} -> 8.2 { $\pm$ 4.2} | Helped |
| Add MCNN constraints to a diffusion behavior-cloning policy | Diff-BC · Franka Kitchen p5 | success rate: 0.02 -> 0.08 | Helped |
| Use tuned rather than fixed MCNN hyperparameters (ablation) | MCNN+MLP · pen-human-v1 | return: 3285 { $\pm$ 209} -> 3405 { $\pm$ 328} | Helped |
| Replace neural-gas memories with randomly chosen memories (ablation) | MCNN+MLP · Adroit human and expert tasks | return | Hurt |
| Increase the number of memories toward 100% of the dataset (ablation) | MCNN+MLP · imitation tasks | return | Hurt |
| Vary the number of memories across the dataset (ablation) | MCNN+MLP · imitation tasks | return | Mixed |
| Increase λ (ablation) | MCNN · MCNN function class | permissible-region width | Helped |
| Increase L (ablation) | MCNN · MCNN function class | function-class width | Hurt |
Surprising findings
- MCNN+MLP outperforms diffusion-based BC on pen-human-v1 despite the MLP being a much simpler architecture.
- Randomly chosen memories instead of neural gas memories cause a significant drop in performance.
Limitations
- MCNNs are only one heuristic way to constrain the function class; better designs may exist.
- Theoretical bound is proven for discrete action spaces, not the continuous settings used in experiments.
- Hyperparameter sensitivity to lambda and number of memories; fixed settings are not always best.
Experimental setup
Datasets: D4RL Adroit (pen, hammer, relocate, door) CARLA (D4RL) Franka Kitchen multimodal relay policy learning dataset
Benchmarks: D4RL Adroit CARLA Town03/Town04 lane Franka Kitchen
Metrics: return normalized score percentage increase in return over D4RL BC p1-p5 task success probabilities
Baselines: D4RL BC MLP-BC 1-NN VINN CQL-Sparse Implicit BC BeT-BC Diff-BC
Builds on: behavior cloning Neural Gas VINN Behavior Transformer Diffusion Policy
Closest papers by meaning
- Imitation Learning by Estimating Expertise of Demonstrators (ICML 2022 · similarity 0.87)
- One-Shot Imitation Learning (NeurIPS 2017 · similarity 0.87)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.86)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.86)
- CCIL: Continuity-Based Data Augmentation for Corrective Imitation Learning (ICLR 2024 · similarity 0.86)
- Latent Diffusion Planning for Imitation Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.