Research map / Reinforcement learning / Imitation learning
GRACE: A Language Model Framework for Explainable Inverse Reinforcement Learning
Silvia Sapora, R Devon Hjelm, Omar Attia, Alexander T Toshev, Bogdan Mazoure
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GRACE rewards instead of GAIL with 2000 trajectories | PPO on BabyAI · BabyAI Multi-task | success rate: 0.31 -> 0.92 | Improved |
| use GRACE rewards instead of GAIL with 2000 trajectories | PPO on BabyAI · BabyAI GoToRedBall | success rate: 0.35 -> 1.00 | Improved |
| use GRACE reward instead of ground-truth sparse reward | PPO on BabyAI · BabyAI OpenMatchingDoor (new) | success rate: 0.79 -> 0.35 | Hurt |
| use GRACE reward instead of ground-truth sparse reward | PPO on BabyAI · BabyAI OpenDoorLoc | success rate: 0.39 -> 1.00 | Improved |
| use GRACE reward instead of ground-truth sparse reward | PPO on BabyAI · BabyAI Multi-task | success rate: 0.95 -> 0.92 | No clear effect |
| replace GPT-4o with Qwen3-Coder-30B as GRACE LLM | GRACE · MuJoCo Ant | average return: 5707 ± 210 -> 6085 ± 804 | No clear effect |
| use GRACE rewards instead of GAIL with 10 trajectories | PPO on MuJoCo · MuJoCo Walker | average return: 790 ± 90 -> 2072 ± 576 | Improved |
| use GRACE (GPT-4o) instead of AIRL with 200 trajectories | PPO on MuJoCo · MuJoCo Ant | average return: 4308 ± 306 -> 5707 ± 210 | Improved |
| use GRACE (GPT-4o) instead of GAIL with 200 trajectories | PPO on MuJoCo · MuJoCo Humanoid | average return: 6521 ± 337 -> 5809 ± 106 | Hurt |
| add targeted reward shaping in Phase 3 (ablation) | GRACE · BabyAI OpenTwoDoors | success rate / learning speed | Improved |
| increase number of expert trajectories to eight (ablation) | GRACE reward recovery · BabyAI | reward accuracy: perfect scores achieved using only eight expert trajectories | Improved |
| add negative trajectories (one negative with enough expert trajectories) (ablation) | GRACE reward recovery · BabyAI | reward accuracy: accuracy of 0.95 with a single negative trajectory | Improved |
Closest papers by meaning
- Imitating Language via Scalable Inverse Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.89)
- REvolve: Reward Evolution with Large Language Models using Human Feedback (ICLR 2025 · similarity 0.89)
- Text2Reward: Reward Shaping with Language Models for Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment (NeurIPS 2024 · similarity 0.88)
- Hybrid Inverse Reinforcement Learning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
