AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Incentivizing LLM Reasoning via Reinforcement Learning with Functional Monte Carlo Tree Search

Kongcheng Zhang, QI YAO, Baisheng Lai, Jiaxing Huang, Wenkai Fang, Dacheng Tao, Mingli Song, Shunyu Liu

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reinforced Functional Token Tuning (RFTT), which adds learnable functional tokens such as <analyze>, <verify>, and <refine> to the LLM vocabulary and uses reinforcement learning with Monte Carlo Tree Search to learn reasoning.

Problem

LLMs need learn-to-reason capabilities, and prior reasoning methods rely on prompting rather than learning the reasoning structure.

Approach

Embeds a set of learnable functional tokens directly into the model vocabulary so the model constructs chain-of-thought through these tokens, with reinforcement fine-tuning guided by functional Monte Carlo Tree Search.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with RFTT (SFT warmup + RL with functional tree search)Qwen-2.5-7B-Instruct · 5 math benchmarks averagePass@1: 66.34 -> 75.66Improved
train with RFTT instead of ReFTQwen-3-4B-Base · 5 math benchmarks averagePass@1: 76.22 -> 79.46Improved
train with RFTT instead of TreeRLLLaMA-3.1-8B-Instruct · MATH-500Pass@1: 57.8 -> 60.2Improved
add RL phase after SFT warmup (ablation)Qwen-2.5-7B-Instruct · 5 math benchmarks averagePass@1: 69.86 -> 75.66Improved
add SFT warmup before RL (ablation)Qwen-2.5-7B-Instruct · 5 math benchmarks averagePass@1: 69.40 -> 75.66Improved
add functional-token MCTS sampling in RL instead of random sampling (ablation)Qwen-2.5-7B-Instruct · 5 math benchmarks averagePass@1: 72.34 -> 75.66Improved
add process reward model in tree search (ablation)LLaMA-3.1-8B-Instruct · 5 math benchmarks averagePass@1: 62.90 -> 64.88Improved
keep <verify> token available (ablation)Qwen-2.5-7B-Instruct · MATH-500accuracy: 72.8% -> 79.8%Improved
keep <subquestion> token available (ablation)Qwen-2.5-7B-Instruct · MATH-500accuracy: 79.4% -> 79.8%No clear effect
add MCTS in RL phaseRFTT training · training costtime per step: 3.2 -> 12.1 min/stepHurt
use RFTT trained on math only on out-of-domain tasksQwen-2.5-7B-Instruct · GPQAaccuracy: 36.4 -> 35.6No clear effect
combine RFTT with DAPOQwen-3-4B-Base · 5 math benchmarks averagePass@1: 79.46 -> 80.78Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy