Research map / Reinforcement learning / Visual and representation RL
Scalable Online Planning via Reinforcement Learning Fine-Tuning
Arnaud Fickinger, Hengyuan Hu, Brandon Amos, Stuart Russell, Noam Brown
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single-agent SPARTA with single-agent RL search | Q-learning blueprint · Hanabi normal | average score: 24.57 -> 24.59 | Improved |
| replace multi-agent SPARTA with multi-agent RL search | Q-learning blueprint · Hanabi normal | average score: 24.61 -> 24.62 | Improved |
| replace multi-agent SPARTA with multi-agent RL search | Q-learning blueprint · Hanabi 2 hints | average score: 23.67 -> 23.76 | Improved |
| replace single-agent SPARTA with single-agent RL search | Q-learning blueprint · Hanabi 2 hints | win rate (%): 25.85 -> 27.85 | Improved |
| replace multi-agent SPARTA with multi-agent RL search | Q-learning blueprint · Hanabi 2 hints | win rate (%): 26.87 -> 31.01 | Improved |
| add multi-agent RL search over blueprint | Q-learning blueprint · Hanabi normal | win rate (%): 63.20 -> 75.93 | Improved |
| use joint (multi-agent) instead of single-agent RL search | RL search · Hanabi 2 hints | win rate (%): 27.85 -> 31.01 | Improved |
| replace MCTS with RL fine-tuning | PPO blueprint · Ms. Pacman | cumulative reward: 5820 -> 8080 | Improved |
| add RL fine-tuning instead of continuing PPO, 8.10^5 samples | weak PPO blueprint · Ms. Pacman | cumulative reward: 1920 -> 5510 | Improved |
| add RL fine-tuning instead of continuing PPO, 8.10^5 samples | random blueprint · Ms. Pacman | cumulative reward: 1280 -> 2730 | Improved |
| amortize MCTS tree over 30 steps instead of replanning each step (ablation) | MCTS · Ms. Pacman | episode return | Hurt |
| use refined value instead of blueprint value for last step (infinite horizon) (ablation) | RL fine-tuning · Ms. Pacman | cumulative reward | No clear effect |
Closest papers by meaning
- Mastering Board Games by External and Internal Planning with Language Models (ICML 2025 · similarity 0.86)
- Bayesian Design Principles for Offline-to-Online Reinforcement Learning (ICML 2024 · similarity 0.85)
- Implicit Search via Discrete Diffusion: A Study on Chess (ICLR 2025 · similarity 0.85)
- Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning (ICML 2025 · similarity 0.85)
- Reinforcement Learning with Multi-Step Lookahead Information Via Adaptive Batching (NeurIPS 2026 · similarity 0.85)
- SUF: Stabilized Unconstrained Fine-Tuning for Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
