Research map / Reinforcement learning / Visual and representation RL
Toward Efficient Exploration by Large Language Model Agents
Dilip Arumugam, Thomas L. Griffiths
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise posterior-sampling temperature above 1 (ablation) | LLM-based PSRL · 5-armed Bernoulli bandit | cumulative regret | Improved |
| use posterior-sampling temperature 1.2 instead of classic Thompson Sampling | LLM-based PSRL · 5-armed Bernoulli bandit (T=100) | cumulative regret | Improved |
| add a well-specified prior in place of an LLM-generated prior (ablation) | LLM-based PSRL · Customer service bandit | cumulative regret | Improved |
| upgrade constituent LLM from GPT-4o to o1-mini (ablation) | LLM-based PSRL · RiverSwim (3 states) | cumulative regret | Improved |
| upgrade constituent LLM from GPT-4o to o1-mini (ablation) | ICRL · RiverSwim (3 states) | cumulative regret | Hurt |
| upgrade constituent LLM from GPT-4o to o1-mini (ablation) | Reflexion · RiverSwim (3 states) | cumulative regret | No clear effect |
| supply all deterministic transitions as prior knowledge (ablation) | LLM-based PSRL · RiverSwim (3 states, GPT-4o) | cumulative regret | Mixed |
| lengthen the river from 3 to 4 states (ablation) | LLM-based PSRL (o1-mini) · RiverSwim | cumulative regret | Hurt |
| use Dirichlet(1,1,1,1) prior instead of fractional parameters (ablation) | LLM-based PSRL · RiverSwim (4 states) | cumulative regret | No clear effect |
| replace GPT-4o with DeepSeek-R1 (ablation) | all LLM agents · Wordle and combination lock | cumulative regret | Improved |
| replace GPT-4o with DeepSeek-R1 in best baseline | Reflexion/ICRL · Wordle | cumulative regret | No clear effect |
| decrease ICRL keep probability p (ablation) | ICRL · Combination lock and Wordle | cumulative regret | Hurt |
Closest papers by meaning
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.92)
- Efficient Reinforcement Learning with Large Language Model Priors (ICLR 2025 · similarity 0.92)
- Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.92)
- EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.91)
- On the Modeling Capabilities of Large Language Models for Sequential Decision Making (ICLR 2025 · similarity 0.91)
- Can large language models explore in-context? (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
