Research map / Large language models / Reasoning and chain of thought

A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MeRF, which injects the natural-language reward specification into the RL training prompt so the model is aware of the optimization objective, improving GRPO-based RLVR on K&K puzzles, math, and CountDown.

Problem

RLVR is sample-inefficient and trial-and-error: the model learns reward patterns only from sparse, fragmented scalar rewards and is blind to the overall reward function during generation.

Approach

Inject the natural-language description of the verifiable reward function (the "rules of the game") into the training prompt as an in-context motivation, so generation is aligned with the optimization objective, while the reward signal remains unchanged.

Key results

• On K&K puzzles, Qwen2.5-7B-Instruct reaches 0.63 average accuracy with MeRF versus 0.51 with RLVR (both evaluated without motivation).
• On math benchmarks with Qwen2.5-7B-Base, MeRF improves average pass@1 by 3.40 points (36.78 vs 33.38) and pass@8 by 4.50 points (54.95 vs 50.45) over RLVR.
• MeRF reaches better pass@4 and pass@8 at step 140 than the final RLVR model at step 280 on K&K puzzles.

Surprising findings

• Adverse (misleading) motivation initially hurts performance, but the model adapts over RL training and learns to discount or reinterpret the misleading motivation.
• Higher sampling temperature increases entropy comparably to MeRF but does not improve performance.

Limitations

• The motivation is static throughout training; dynamic motivation is not explored.
• Gains are limited for models with weak generalization capability, where RLVR itself is hard to make efficient.
• Training and validation prompts differ (motivation present only in training), a gap the authors found to be negligible but which is a design constraint.

Experimental setup

Datasets: K&K logic puzzles DAPO-Math-17K CountDown

Benchmarks: K&K logic puzzles (2-8 people) AIME24 AIME25 AMC23 MATH500 CountDown Natural Questions (search agent)

Metrics: accuracy pass@1 pass@2 pass@4 pass@8 entropy

Baselines: RLVR (GRPO without motivation) o3-mini-high o1-2024-12-17 DeepSeek-R1 GPT-4o GPT-4o-mini

Builds on: GRPO RLVR DeepSeek-R1 Logic-RL

Closest papers by meaning

• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.92)
• Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning (ICML 2026 · similarity 0.91)
• From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation (ICLR 2026 · similarity 0.91)
• The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.91)
• Learning to Reason without External Rewards (ICLR 2026 · similarity 0.91)
• RM-R1: Reward Modeling as Reasoning (ICLR 2026 · similarity 0.91)