Research map / Reinforcement learning / Imitation learning
Large Language Models can Implement Policy Iteration
Ethan Brooks, Logan A Walls, Richard Lewis, Satinder Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LLM-rollout Q-values with argmax policy improvement instead of the No argmax imitation baseline | ICPI · six illustrative domains | normalized regret | Improved |
| ICPI instead of tabular Q-learning | ICPI · six illustrative domains | normalized regret | Improved |
| ICPI instead of nearest-neighbor trajectory matching model | ICPI · six illustrative domains | normalized regret | Improved |
| remove hints from the prompt format (ablation) | ICPI · six illustrative domains | normalized regret | Mixed |
| remove balancing of terminal and reward time-steps in prompts (ablation) | ICPI · six illustrative domains | normalized regret | Mixed |
| remove constraints on prompt time-steps (ablation) | ICPI · six illustrative domains | normalized regret | Mixed |
| double recency cutoff from 8 to 16 trajectories (ablation) | ICPI · six illustrative domains | normalized regret | Mixed |
| replace Codex with smaller models GPT-J, InCoder, OPT-30B, code-cushman-001 | ICPI · six illustrative domains | normalized regret | Hurt |
Closest papers by meaning
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.89)
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.89)
- Controlling Large Language Model with Latent Action (ICML 2025 · similarity 0.89)
- MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions (ICLR 2025 · similarity 0.89)
- Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.89)
- d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
