Research map / Large language models / Agents, code and math
When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training
Sanxing Chen, Xiaoyin Chen, Yukun Huang, Roy Xie, Bhuwan Dhingra
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL or SFT training on bandit tasks | Qwen2.5-7B-Instruct · Gaussian5_Var1_MeanN0 (in-distribution) | AvgReward@300: 0.79 -> 1.01 to 1.05 | Improved |
| use algorithmic (UCB-imitation) reward instead of raw bandit reward (ablation) | Qwen2.5-7B RL · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 79.8 -> 85.7 | Improved |
| use strategic regret-shaped reward instead of raw reward (ablation) | Qwen2.5-7B RL · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 79.8 -> 81.1 | Improved |
| use strategic reward in low-variance Bernoulli training (ablation) | Qwen2.5-7B RL · Bernoulli5_Uniform | regret | No clear effect |
| use strategic reward on 3B model instead of raw reward (ablation) | Qwen2.5-3B RL · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 49.1 -> 56.9 | Improved |
| RL on task reward without teacher | Qwen2.5-3B RL-OG · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 32.7 -> 49.1 | Mixed |
| add teacher guidance (RL-ALG or SFT) for 3B model (ablation) | Qwen2.5-3B · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 49.1 (RL-OG) -> 71.2 (RL-ALG), 80.8 (SFT) | Improved |
| train with RL/SFT (vs pretrained) | Qwen2.5-7B · Gaussian5_Var1_MeanN0 (in-distribution) | SuffixFail@150 (%): 0.0 -> 4.7 to 9.4 | Hurt |
| imitate UCB with RL-ALG | Qwen2.5-7B vs UCB teacher · Gaussian5_Var1_MeanN0 (in-distribution) | BestArmFreq@300 (%): 80.6 -> 85.7 | Improved |
| SFT trained on Bernoulli, evaluated on Gaussian | Qwen2.5 SFT · Gaussian5_Var1_MeanN0 (OOD) | worst-case regret | Hurt |
| use RL-ALG instead of SFT for OOD transfer (ablation) | Qwen2.5-7B · cross-distribution | worst-case regret | Improved |
| train in low variance, test on harder high-variance MeanU environments | learned policies · Gaussian5_VarX_MeanU | regret | Hurt |
Closest papers by meaning
- From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems (ICML 2024 · similarity 0.91)
- EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.90)
- Do LLM Agents Have Regret? A Case Study in Online Learning and Games (ICLR 2025 · similarity 0.90)
- Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward (ICLR 2026 · similarity 0.89)
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use (ICML 2026 · similarity 0.89)
- On the Modeling Capabilities of Large Language Models for Sequential Decision Making (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
