Research map / Large language models / Reasoning and chain of thought

RAST: Reasoning Activation in LLMs via Small-model Transfer

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RAST, a decoding-time method that adds the logit difference between a small RL-trained model and its base to a larger base model, recovering much of the RL gain without RL on the large model.

Problem

Applying RL to large LLMs for reasoning is resource-intensive (multiple model copies, heavy GPU memory). The paper asks whether RL-induced output probability shifts are transferable across model scales.

Approach

RL mainly reshapes output distributions to elicit latent reasoning behaviors, so the logit delta (RL-trained small model minus its base) can be injected at decoding time into a larger base model: softmax[M_base + λ(S_RL − S_base)], with no training of the large model.

Key results

• Applying ΔR_14B from a 14B RL model to Qwen-2.5-32B lifts MATH500 from 68.6 to 80.7 (SimpleRL-32B ceiling 81.3) with 95.3% recovery on GSM8K-style gap analysis, at roughly 160G vs 350G GPU memory.
• Llama-3.1-70B with ΔR_8B improves MATH500 from 26.2 to 33.1 and GSM8K from 56.1 to 82.2.
• On code, Qwen-2.5-14B-1M with ΔR_7B raises average over HumanEval+, MBPP+, LiveCodeBench from 62.1 to 66.5.

Measured effects reported in the paper

Surprising findings

• RAST with ΔR_14B on the 32B base sometimes matches or exceeds the 32B RL-trained model on MATH500, Minerva and GSM8K.
• Ensembling ΔR from 14B, 7B and 1.5B experts (about 75.6 on MATH500) did not beat the single ΔR_14B.

Limitations

• Gains on hard benchmarks such as AIME are modest.
• Requires a reasonably strong small RL-trained expert, which itself costs RL training.
• Lacks theoretical grounding for why logit deltas transfer across scales.

Experimental setup

Datasets: MATH GSM8K MATH500 Minerva OlympiadBench AIME24 AMC23 HumanEval+ MBPP+ LiveCodeBench

Benchmarks: MATH500 AIME24 AMC Minerva OlympiadBench GSM8K HumanEval+ MBPP+ LiveCodeBench

Metrics: pass@1 averaged over 32 runs recovery rate pass@k majority@k cosine similarity KL divergence GPU memory

Baselines: SimpleRL-32B SimpleRL-14B SimpleRL-7B Qwen-2.5-32B base Llama-3.1-70B base Code-R1-Zero

Builds on: GRPO SimpleRL-Zoo contrastive decoding vLLM

Closest papers by meaning

• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.93)
• RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.92)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.92)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.92)
• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.91)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.91)