Research map / Reinforcement learning / Visual and representation RL
The State of Reinforcement Finetuning for Transformer-based Generative Agents
Shengchao Hu, Peng Wang, Guozheng Ma, Shi Fu, Li Shen, Ya Zhang, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Q-guided policy improvement term to SFT (QP-SFT) (ablation) | SFT, Fullmodel tuning · Meta-RL average (expert) | average return: 336.07 -> 372.09 | Improved |
| add Q-guided policy improvement term to SFT (QP-SFT) (ablation) | SFT, Decorator tuning · Meta-RL average (expert) | average return: 321.73 -> 334.16 | Improved |
| add Q-guided policy improvement term to SFT (QP-SFT) (ablation) | SFT, Adaptor tuning · Meta-RL average (expert) | average return: 331.69 -> 335.87 | Improved |
| add Q-guided policy improvement term to SFT (QP-SFT) (ablation) | SFT, Prompt tuning · Meta-RL average (expert) | average return: 315.05 -> 317.07 | Improved |
| add Q-guided policy improvement term to DPO (QP-DPO) (ablation) | DPO, Fullmodel tuning · Meta-RL average (expert) | average return: 326.83 -> 362.13 | Improved |
| add Q-guided policy improvement term to DPO (QP-DPO) (ablation) | DPO, Decorator tuning · Meta-RL average (expert) | average return: 314.66 -> 330.08 | Improved |
| add Q-guided policy improvement term to DPO (QP-DPO) (ablation) | DPO, Adaptor tuning · Meta-RL average (expert) | average return: 333.38 -> 339.46 | Improved |
| add Q-guided policy improvement term to DPO (QP-DPO) (ablation) | DPO, Prompt tuning · Meta-RL average (expert) | average return: 313.73 -> 318.39 | Improved |
| add Q-guided term to SFT on MetaWorld with full-model tuning (ablation) | SFT · MetaWorld (expert, Fullmodel tuning) | return: 441.05 -> 553.36 | Improved |
| switch finetuning from Adaptor to Decorator parameters (ablation) | PPO · MetaWorld | performance: 3% gain | Improved |
| switch algorithm from PPO to QP-DPO (ablation) | PPO with Adaptor · MetaWorld | performance: 10% improvement | Improved |
| increase pretrained agent size with full-model finetuning (12.66M to 49.32M trainable) (ablation) | QP-DPO Fullmodel · MetaWorld (medium data) | performance: 694.87 -> 375.22 | Hurt |
Closest papers by meaning
- Vanishing Gradients in Reinforcement Finetuning of Language Models (ICLR 2024 · similarity 0.89)
- SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents (NeurIPS 2025 · similarity 0.89)
- BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning (ICLR 2026 · similarity 0.88)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.88)
- Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models (ICLR 2026 · similarity 0.88)
- Behavior Injection: Preparing Language Models for Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
