AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

Does Reinforcement Fine-Tuning Improve Generalization of LLM Agents? An Empirical Study

Zhiheng Xi, Xin Guo, Jiaqi Liu, Jiazheng Zhang, Yutao Fan, Zhihao Zhang, Shichun Liu, Mingxu Chai, Xiaowei Shi, Yitao Zhai, Xunliang Cai, Tao Gui and 2 more

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Empirical study of whether reinforcement fine-tuning (RFT) of LLM agents generalizes to unseen environments that differ in background knowledge, observation spaces, and action spaces.

Problem

Most RFT evaluations for multi-turn LLM agents are in-domain (same environment or same tasks), leaving out-of-domain generalization to unseen environments with different knowledge, observations, and actions unexplored.

Approach

Systematically evaluates RFT-trained agents on unseen environments that vary background knowledge, observation space, and action space, rather than only in-domain test tasks; the abstract does not specify the method details.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with easy tasks, test on hard tasksRFT Qwen2.5-7B-Instruct · WebShop (hard)avg@8: 17.4 -> 77.5Improved
train on hard tasks instead of easy tasksRFT Qwen2.5-3B-Instruct · AlfWorldavg@8: +8.7 pointsImproved
easy-to-hard curriculum instead of single difficulty levelRFT Qwen2.5-3B-Instruct · BabyAIavg@8: +2.0 and +3.3 pointsImproved
easy-to-hard curriculum instead of mixed training on all tasksRFT Qwen2.5-3B-Instruct · WebShopavg@8: 87.9 -> 88.7Improved
easy-to-hard curriculum instead of mixed training on all tasksRFT Qwen2.5-7B-Instruct · WebShopavg@8: 86.5 -> 83.0Hurt
RFT on easy BabyAI tasks reduces interaction turnsRFT Qwen2.5-3B-Instruct · BabyAIaverage interaction turns: 10.76 -> 4.19Improved
train on SearchQA, evaluate on WebShop (unseen)RFT Qwen2.5-7B-Instruct · WebShopavg@8: +18.48 pointsImproved
train on BabyAI, evaluate on WebShop (unseen)RFT Qwen2.5-7B-Instruct · WebShopavg@8: 28.59 -> 10.25Hurt
RFT on single environment evaluated on unseen environments (average)RFT Qwen2.5-3B-Instruct and 7B · Held-out environmentsavg@8: +3.32 and +3.44 pointsImproved
sequentially train on TextCraft after WebShopRFT Qwen2.5-7B-Instruct · TextCraft (downstream)avg@8: 80.88 -> 82.50Improved
sequentially train on TextCraft after WebShop, check upstream forgettingRFT Qwen2.5-7B-Instruct · WebShop (upstream)avg@8: 86.5 -> 86.32No clear effect
train BabyAI after TextCraft (TC+BA vs TC)RFT Qwen2.5-7B-Instruct · WebShop (unseen)avg@8: 38.30 -> 23.0Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy