AI Paper Atlas

Research map / Reinforcement learning / Imitation learning

PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning

Angelos Filos, Clare Lyle, Yarin Gal, Sergey Levine, Natasha Jaques, Gregory Farquhar

ICML 2021 · Reinforcement learning › Imitation learning · approach: Reward learning from demonstrations · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use ITD to infer rewards from reward-free demonstrationsDQN on Roundabout · Roundaboutnormalized return: 0.85±0.02 -> 0.92±0.01Improved
Use ITD to infer rewards from reward-free demonstrationsDQN on CoinGrid · CoinGridnormalized return: 0.73±0.02 -> 0.77±0.03Improved
Use ITD to infer rewards from reward-free demonstrationsPPO on FruitBot · FruitBotnormalized return: 0.37±0.02 -> 0.35±0.04Hurt
Use ΨΦ-learning for 0-shot task transferCoinGrid · R-G and -R+Greturn: 0.0±0.0 -> 0.2±0.1Improved
Use ΨΦ-learning for 0-shot task transferCoinGrid · -R-Greturn: -1.0±0.0 -> -0.4±0.2Improved
Adapt using one episode of new-task experienceΨΦ-learning on CoinGrid · R-G, -R+G, and -R-Greturn: 0.0±0.0 -> 1.0±0.0; -1.0±0.0 -> 1.0±0.0Improved
Add an L1 sparsity penalty on preferences (ablation)ITD · Offline multi-task IRLoverall performanceNo clear effect
Use RL interaction to update shared cumulants alongside ITD (ablation)ΨΦ-learning imitation model · Roundabout held-out demonstrationsaction prediction accuracy: little over 80% -> ≈95%Improved
Use demonstrations that are irrelevant or misleading rather than only ego-task dataΨΦ-learning · Highway adversarial taskperformanceNo clear effect
Combine multi-task demonstrations with online RL and GPIΨΦ-learning · Highway multi-taskperformanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy