Research map / Large language models / Agents, code and math
Large Language Models Think Too Fast To Explore Effectively
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Evaluates whether LLMs explore effectively in the open-ended Little Alchemy 2 game versus 29,493 humans, finds most LLMs underperform except o1, and uses regression, reasoning-trace analysis, and sparse autoencoders to show uncertainty signals appear earlier than empowerment signals in LLaMA3.1-70B.
Problem
Whether LLMs can explore effectively in open-ended tasks comparable to humans, what exploration strategies they use (uncertainty-driven vs. empowerment), and what internal mechanisms underlie their success or failure.
Approach
Adapts the Little Alchemy 2 human paradigm to LLMs, separating uncertainty-driven and empowerment-based choice weights via regression, compares reasoning traces of GPT-4o and DeepSeek-R1, and probes layerwise representations with sparse autoencoders to argue LLMs 'think too fast' because uncertainty/choice signals arise before empowerment signals.
Key results
- o1 discovered 177 elements on average in 500 trials versus 42 for humans, the only LLM to significantly outperform humans; LLaMA3.1-8B discovered 9.
- DeepSeek-R1 discovered 85 elements and produced much longer reasoning traces than GPT-4o, using all seven reasoning labels.
- In LLaMA3.1-70B, uncertainty correlated with the representation at layer 2 (r=0.73) while empowerment peaked at layer 72 (r=0.55), and choice beta weights peaked at layer 1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use o1 instead of humans | Little Alchemy 2 exploration · Little Alchemy 2, 500 trials | new elements discovered: 42 -> 177 | Helped |
| use DeepSeek-R1 instead of humans | Little Alchemy 2 exploration · Little Alchemy 2, 500 trials | new elements discovered: 42 -> 85 | Helped |
| use GPT-4o instead of humans | Little Alchemy 2 exploration · Little Alchemy 2, 500 trials | new elements discovered: 42 -> 35 | Hurt |
| scale LLaMA3.1 from 8B to 70B | LLaMA3.1 · Little Alchemy 2, 500 trials | new elements discovered: 9 -> 25 | Helped |
| increase sampling temperature | LLMs · Little Alchemy 2 | performance: β = 0.124, z = 5.060, p < 0.001 | Helped |
| increase sampling temperature | LLMs · Little Alchemy 2 | uncertainty-driven strategy use: β = 2.911, z = 7.440, p < 0.001 | Helped |
| increase sampling temperature | LLMs · Little Alchemy 2 | empowerment strategy use: β = 0.196, z = 1.067, p = 0.286 | No clear effect |
| add CoT, self-reflection, and explicit strategy hints (ablation) | GPT-4o at temperature = 1 · Little Alchemy 2 | average inventory: average inventory = 43, t = -1.17, p = 0.304 | No clear effect |
| ablate the SAE-identified empowerment neuron (ablation) | LLaMA3.1-70B · Little Alchemy 2 | empowerment strategy use | Hurt |
| ablate the SAE-identified uncertainty neuron (ablation) | LLaMA3.1-70B · Little Alchemy 2 | task performance | Hurt |
| intervene on the uncertainty neuron at factors other than 1.0 (ablation) | LLaMA3.1-70B · Little Alchemy 2 | average inventory: 0.0, 0.5, 0.7, 1.0, 2.0 | Hurt |
| set the empowerment-layer intervention to a value other than 1.5 (ablation) | LLaMA3.1-70B · Little Alchemy 2 | average inventory: 1.5 | Hurt |
Surprising findings
- Higher sampling temperature improved LLM performance only moderately and mainly increased uncertainty-driven choices rather than empowerment-driven ones.
- Prompt engineering with explicit exploration hints did not significantly improve GPT-4o performance (43 elements, p=0.304).
Limitations
- Underlying cause of LLMs 'thinking too fast' remains unclear and is not established causally.
- Empowerment is estimated with a neural network from prior work and the paper's dynamic update rule, so measures depend on that proxy.
- Only a small set of models were tested, with SAE analysis restricted mainly to LLaMA3.1-70B and 8B.
Experimental setup
Datasets: Little Alchemy 2 human trial data (29,493 participants, 4,691,033 trials)
Benchmarks: Little Alchemy 2
Metrics: average number of discovered elements regression weights for empowerment and uncertainty SAE correlation with choices/uncertainty/empowerment reasoning label token usage
Baselines: GPT-4o LLaMA3.1-8B LLaMA3.1-70B human participants
Builds on: Brändle et al. open-ended exploration paradigm sparse autoencoders chain-of-thought prompting
Closest papers by meaning
- Can large language models explore in-context? (NeurIPS 2024 · similarity 0.92)
- Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Go (NeurIPS 2025 · similarity 0.90)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.90)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.90)
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.90)
- EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.