Research map / Large language models / Instruction and fine-tuning
Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs
Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL fine-tuning (PPO/GRPO) | SFT 7B math models · MATH/GSM8K/College Math | activation intensity: Mistral MATH alpha=0.5: 3.58e-4 -> 4.05e-4 | Improved |
| online RL fine-tuning | SFT models · MATH, GSM8K | information complexity: GSM8K Deepseek-Math alpha=0.5: 1.37e-1 -> 3.23e-1 | Improved |
| online RL fine-tuning | SFT models · MATH | distribution kurtosis: Deepseek-Math alpha=0.5: 3.03e+2 -> 1.89e+2 | Improved |
| DPO instead of online RL | Qwen2.5-7B-SFT · MATH, College Math | activation intensity / information complexity: MATH alpha=0.1 intensity: 6.95e-4 -> 6.90e-4 | No clear effect |
| DPO fine-tuning | Qwen2.5-7B-SFT · all datasets | distribution kurtosis: MATH alpha=0.5 not reported; alpha=0.1: 5.44e+2 -> 4.83e+2 | Improved |
| lower sampling temperature (0.6 vs 1.0) during RL (ablation) | GRPO variant on Qwen2.5-3B-Instruct · GSM8K test | accuracy: 36.5 -> 51.2 | Improved |
| RL post-training | Mistral-7B SFT · Minerva math | accuracy: 12.1 -> 11.8 | Hurt |
| RL post-training | DeepSeek-Math, DS-Distill-Qwen · MATH | accuracy: 46.2 -> 52.6; 88.4 -> 95.4 | Improved |
Closest papers by meaning
- Understanding the Effects of RLHF on LLM Generalisation and Diversity (ICLR 2024 · similarity 0.91)
- From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones (ICLR 2026 · similarity 0.90)
- Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data (ICML 2024 · similarity 0.90)
- Angles Don’t Lie: Unlocking Training‑Efficient RL Through the Model’s Own Signals (NeurIPS 2025 · similarity 0.90)
- Interpreting Learned Feedback Patterns in Large Language Models (NeurIPS 2024 · similarity 0.90)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
