Research map / Large language models / Reasoning and chain of thought

RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RAIN-Merging, a gradient-free method that merges an instruction-tuned model's task vector into a large reasoning model via null-space projection on thinking-token features plus instruction-attention-guided per-module scaling.

Problem

Large reasoning models (LRMs) with explicit thinking segments often fail to follow instructions on output format and constraints, while SFT to fix this is costly and can cause capability regressions; naive task-vector merging with instruction-tuned models can break the thinking format.

Approach

Project the instruction-tuned model's task vector onto the null space of forward features at thinking special tokens (so the thinking-token output distribution is preserved), then scale per-module coefficients using instruction-attention alignment minus leakage, computed in closed form from forward attention statistics.

Key results

• On DeepSeek-R1-Distill-Qwen-7B, RAIN-Merging reaches 48.11 average instruction-following and 55.59 reasoning/general average, versus 44.12 and 51.03 for the LRM.
• Task Arithmetic yields 6.4% missing </think> tokens and thinking-token KL 0.1224, while RAIN-Merging gives 0.0% missing and KL 0.0065.
• Across scales, relative average gains over the LRM are +8.20% (1.5B), +6.17% (14B), +1.57% (32B) and +7.78% (Llama-3.1-8B on reasoning-general side reported as +7.78%).

Surprising findings

• Merging the instruction-tuned model improves reasoning and general benchmarks (e.g., Math and GPQA) as well as instruction following, which the authors attribute to better internal chain-of-thought.
• Naive Task Arithmetic merging breaks thinking-format termination, leaving 6.4% of outputs without a closing </think> tag.

Limitations

• Merged model still trails the instruction-tuned model slightly on instruction following.
• Gains on 32B model are small (+1.57% average instruction following).
• Relies on calibration sets and forward-feature storage, increasing memory versus data-free merging.

Experimental setup

Datasets: Mixture-of-Thoughts IFEval MathIF

Benchmarks: IFEval CELLO InfoBench ComplexBench Math GPQA Aider Arena-Hard-v2 ALFWorld WebShop MathIF AIME25

Metrics: accuracy average score relative gain run-time (minutes) KL divergence at thinking tokens missing </think> rate RIC RAA

Baselines: Task Arithmetic SLERP Karcher TIES DARE-TIES ACM-TIES LEWIS-TIES AIM-TIES SFT

Builds on: task arithmetic TIES-Merging AIM AlphaEdit gradient projection memory

Closest papers by meaning

• When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs (NeurIPS 2025 · similarity 0.89)
• Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.89)
• Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following (AAAI 2026 · similarity 0.89)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.88)
• On Reasoning Strength Planning in Large Reasoning Models (NeurIPS 2025 · similarity 0.88)
• LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning (NeurIPS 2026 · similarity 0.88)