Research map / Reinforcement learning / Imitation learning

Adversarial Imitation Learning via Boosting

ICLR 2024 · Reinforcement learning › Imitation learning · approach: Adversarial imitation learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AILBoost, a fully off-policy adversarial imitation learning method that performs gradient boosting in state-action occupancy space, training discriminators on a weighted replay buffer of all past policies' data.

Problem

Adversarial imitation learning is on-policy and sample-inefficient; off-policy variants like DAC lack a guarantee of distribution matching with the expert, while ValueDICE's off-policy objective is biased under stochastic transitions.

Approach

Treat learned policies as weak learners in a gradient-boosting scheme minimizing reverse KL between the ensemble's occupancy measure and the expert's; the discriminator is trained on a replay buffer weighted by ensemble mixture weights (older policies discounted), and each weak learner is trained with SAC on the discriminator reward.

Key results

• AILBoost outperforms DAC on both state-based and pixel-based DMC environments, with comparable to better performance on Walker Walk and Cheetah Run from images.
• On state-based DMC, AILBoost outperforms ValueDICE and IQ-Learn and reaches expert performance with as little as one expert trajectory.
• Probability of AILBoost outperforming DAC is approximately 50%, though its IQM and mean improvements over DAC are larger.

Measured effects reported in the paper

Surprising findings

• Probability of improvement over DAC is about 50% even though IQM and mean gains are larger, indicating the gains are in magnitude rather than consistency across tasks.
• More policy updates per discriminator update makes AILBoost significantly less sample efficient, though it asymptotically reaches expert performance.

Limitations

• Maintains an ensemble of policies, increasing memory cost relative to DAC.
• Discriminator update schedule is sensitive; over-optimizing the policy per discriminator update hurts sample efficiency.
• Evaluated only on DeepMind Control Suite; discrete control and learning from observations left to future work.

Experimental setup

Datasets: DeepMind Control Suite expert demonstrations

Benchmarks: DeepMind Control Suite: Ball in Cup Catch, Walker Walk, Cheetah Run, Quadruped Walk, Humanoid Stand DeepMind Control Suite vision: Walker Walk, Cheetah Run

Metrics: inter-quartile mean mean optimality gap probability of improvement

Baselines: Behavior cloning ValueDICE DAC IQ-Learn

Builds on: Discriminator Actor Critic ValueDICE GAIL AIRL SAC

Closest papers by meaning

• Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (ICLR 2019 · similarity 0.91)
• Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency (AAAI 2022 · similarity 0.91)
• On Discovering Algorithms for Adversarial Imitation Learning (ICLR 2026 · similarity 0.91)
• Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation (NeurIPS 2024 · similarity 0.89)
• DiffAIL: Diffusion Adversarial Imitation Learning (AAAI 2024 · similarity 0.88)
• Diffusing States and Matching Scores: A New Framework for Imitation Learning (ICLR 2025 · similarity 0.88)