Research map / Reinforcement learning / Imitation learning
When Demonstrations meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
Siliang Zeng, Chenliang Li, Alfredo Garcia, Mingyi Hong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train world model on medium-expert instead of medium transitions | Offline ML-IRL · D4RL halfcheetah, 5,000 demos | average episode reward: 7640.73 -> 11231.40 | Improved |
| train world model on medium-replay instead of medium transitions | Offline ML-IRL · D4RL halfcheetah, 5,000 demos | average episode reward: 7640.73 -> 9236.84 | Improved |
| Offline ML-IRL versus CLARE (also model-based offline IRL) | Offline ML-IRL · D4RL halfcheetah medium-exp, 5,000 demos | average episode reward: 622.79 -> 11231.40 | Improved |
| Offline ML-IRL versus BC on halfcheetah medium | Offline ML-IRL · D4RL halfcheetah medium, 5,000 demos | average episode reward: 4471.72 -> 7640.73 | Improved |
| Offline ML-IRL versus ValueDICE on hopper medium | Offline ML-IRL · D4RL hopper medium, 5,000 demos | average episode reward: 3073.16 -> 2453.25 | Hurt |
| reduce expert demonstrations from 5,000 to 1,000 | Offline ML-IRL · D4RL halfcheetah medium | average episode reward: 7640.73 -> 7690.30 | No clear effect |
| reduce expert demonstrations from 5,000 to 1,000 | BC · D4RL halfcheetah medium | average episode reward: 4471.72 -> 2799.40 | Hurt |
| reduce expert demonstrations from 5,000 to 1,000 | ValueDICE · D4RL walker2d | average episode reward: 3191.47 -> 1794.97 | Hurt |
| increase expert demonstrations from 5,000 to 10,000 | Offline ML-IRL · D4RL walker2d medium-exp | average episode reward: 4201.40 -> 4363.54 | No clear effect |
| increase expert demonstrations from 5,000 to 10,000 | BC · D4RL hopper | average episode reward: 2801.19 -> 3236.70 | Improved |
| transfer reward learned on medium-expert to medium-replay and run MOPO | Offline ML-IRL · D4RL medium-replay offline RL | average episode reward | No clear effect |
| exclude IQ-Learn from comparison | IQ-Learn · D4RL | average episode reward | Hurt |
Closest papers by meaning
- Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms (ICML 2024 · similarity 0.93)
- Recent Advancements in Inverse Reinforcement Learning (AAAI 2024 · similarity 0.91)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.91)
- Inverse Reinforcement Learning by Estimating Expertise of Demonstrators (AAAI 2025 · similarity 0.90)
- Reward-free World Models for Online Imitation Learning (ICML 2025 · similarity 0.90)
- Sub-optimal Experts mitigate Ambiguity in Inverse Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
