Research map / Large language models / Instruction and fine-tuning
Breaking Barriers: Do Reinforcement Fine-tuning Gains Transfer To Unseen Domains?
Chuxuan Hu, Yuxuan Zhu, Antony Kellermann, Caleb Biddulph, Suppakit Waiwitlikhit, Jason Benn, Daniel Kang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply RPT on in-domain versus out-of-domain tasks | 18 open RPT models · Observational study | pass@1 gain: 2.87 in-domain vs -3.19 out-of-domain | Mixed |
| add single-domain RPT on math, evaluated out-of-domain | DeepSeek-R1-Distill-Qwen-1.5B + GRPO · OOD tasks | pass@1 gain: 3.13 in-domain -> -1.81 out-of-domain | Hurt |
| train with math RPT, evaluated on code | math-RPT models · Observational study, code tasks | pass@1: 4.77% improvement on code | Improved |
| train with code RPT, evaluated on math | code-RPT models · Observational study, math tasks | pass@1: 9.49 on code, 15.44 on math | Improved |
| train with math or code RPT, evaluated on knowledge tasks | math/code-RPT models · Observational study, knowledge tasks | pass@1: -0.27 vs 11.08 on math and 5.82 on code | No clear effect |
| train with knowledge RPT, evaluated on math and code | knowledge-RPT models · Observational study | pass@1: 21.40 on math, 12.16 on code | Improved |
| train with knowledge RPT, evaluated on other knowledge subdomains | Fino1-8B · Observational study | TabFact accuracy: 64.18% -> 48.39% | Hurt |
| switch RL algorithm from GRPO to DAPO (ablation) | DeepSeek-R1-Distill-Qwen-1.5B math RPT · Table 3 | ID-OOD gain gap: 4.94 -> 5.23 | No clear effect |
| switch base model from DeepSeek-R1-Distill-Qwen-1.5B to Llama-3.2-3B-Instruct (ablation) | math RPT with GRPO · Table 3 | ID-OOD gain gap: 4.94 -> 5.06 | No clear effect |
| increase RPT training from 0.5 to 2 epochs (ablation) | Math-RPT · Figure 5 | OOD pass@1 gain: -0.63 -> -1.47 OOD; 3.19 -> 4.72 ID | Hurt |
| scale up model size within a family (ablation) | open RPT models · Table 4 | ID-OOD gain gap: 1.58 average gap difference (Large - Small) | No clear effect |
| train on mixed-domain data instead of the best single domain (ablation) | Mixed-RPT · Table 5 | pass@1 gain on knowledge and code: knowledge -3.98 -> -2.17; code -0.02 -> 1.91; math -0.15 -> -3.62 | Mixed |
Closest papers by meaning
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.90)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.90)
- Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs (ICLR 2026 · similarity 0.89)
- Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead (ICLR 2026 · similarity 0.89)
- Behavior Injection: Preparing Language Models for Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- UFT: Unifying Supervised and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
