Research map / Reinforcement learning / RL from feedback
GTR-Turbo: Merged Checkpoint is Secretly a Free Teacher for Agentic VLM Training
Tong Wei, Yijun Yang, Changhao Zhang, Junliang Xing, Yuanchun Shi, Zongqing Lu, Deheng Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GPT-4o corrector with merged-checkpoint teacher (KL guidance) | GTR (Qwen2.5-VL-7B) · Points24 | success rate (%): 44.5 -> 53.5 | Improved |
| use SFT guidance from merged teacher instead of GPT-4o corrector | GTR (Qwen2.5-VL-7B) · Points24 | success rate (%): 44.5 -> 48.0 | Improved |
| use soft-logit KL guidance instead of SFT guidance | GTR-Turbo · Points24 | success rate (%): 48.0 -> 53.5 | Improved |
| use KL guidance instead of GPT-4o corrector (training time) | GTR · Points24 | training time (hours): 191h -> 89h | Improved |
| use KL guidance instead of GPT-4o corrector (cost) | GTR · Points24 | additional cost (USD): $307.78 -> $114.81 | Improved |
| use merged-checkpoint KL teacher instead of GPT-4o corrector | GTR · ALFWorld | average success rate: 0.16 -> 0.15 | No clear effect |
| use SFT guidance from merged teacher instead of GPT-4o corrector | GTR · ALFWorld | average success rate: 0.16 -> 0.12 | Hurt |
| use merged teacher instead of static base-model KL reference (ablation) | GTR-Turbo · Points24 | success rate | Improved |
| use RL exploration instead of rejection sampling (ablation) | self-improvement baseline · Points24 | success rate | Improved |
| guide full response (thought + action) instead of thought only (ablation) | GTR-Turbo · Points24 | success rate | Hurt |
| use TIES merging instead of linear averaging (ablation) | GTR-Turbo · Points24 | success rate | Improved |
| clip negative KL estimates instead of raw K1 (ablation) | GTR-Turbo (KL) · Points24 | success rate | Improved |
Closest papers by meaning
- Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.89)
- Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs (CVPR 2026 · similarity 0.89)
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents (NeurIPS 2025 · similarity 0.89)
- VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Empowering Small VLMs to Think with Dynamic Memorization and Exploration (ICLR 2026 · similarity 0.88)
- COVR: Collaborative Optimization of VLMs and RL Agent for Visual-Based Control (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
