Research map / Reinforcement learning / Visual and representation RL
On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning
Zhao Mandi, Pieter Abbeel, Stephen James
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| multi-task pretraining + fine-tuning instead of Reptile meta-pretraining | PPO on Procgen Coinrun · unseen Coinrun levels | return after adaptation | Improved |
| RL2 test-time adaptation instead of fine-tuning | PPO on Procgen Coinrun · unseen Coinrun levels | return after adaptation | Hurt |
| increase number of pretraining levels (100 to 10000) (ablation) | MT-PPO · Procgen Coinrun unseen levels | zero-shot and fine-tuned return | Improved |
| multi-task fine-tuning vs Reptile meta-pretraining | C2F-ARM · RLBench 5 unseen tasks, 0 demos | success rate | No clear effect |
| multi-task fine-tuning vs PEARL context adaptation | C2F-ARM · RLBench 5 unseen tasks | success rate | Improved |
| multi-task pretraining then fine-tuning instead of training from scratch | C2F-ARM · RLBench unseen tasks, sparse reward | success rate | Improved |
| fine-tune unseen task in isolation instead of jointly with training tasks (ablation) | MT-C2F-ARM · RLBench unseen tasks | success rate | Improved |
| fine-tune both Q-attention depths instead of only the fine depth (ablation) | MT-C2F-ARM · RLBench unseen tasks | success rate | Improved |
| multi-task pretraining then fine-tuning vs training from scratch | Rainbow DQN · Atari 5 unseen games | normalized score | No clear effect |
| fine-tune only convolutional layers (reinit MLP) instead of all parameters (ablation) | MT-Rainbow · Atari unseen games | normalized score | No clear effect |
| add test-time fine-tuning to PEARL-Rainbow (ablation) | PEARL-Rainbow · Atari unseen games | normalized score | Improved |
| PEARL gradient-free context adaptation | C2F-ARM/Rainbow · RLBench and Atari unseen tasks | adaptation performance | Hurt |
Closest papers by meaning
- AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.92)
- MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.91)
- Meta-World+: An Improved, Standardized, RL Benchmark (NeurIPS 2025 · similarity 0.91)
- Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator (NeurIPS 2024 · similarity 0.90)
- DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
- Meta-Reinforcement Learning via Exploratory Task Clustering (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
