Research map / Large language models / Reasoning and chain of thought
Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAIF, an RLVR recipe (GRPO with rule-centric rewards, superior-CoT replay filtering, and behavior cloning) on an LLM-evolved complex-instruction dataset to make LLMs reason before following complex, compositional instructions.
Problem
Vanilla chain-of-thought hurts instruction following under complex compositional constraints because models produce superficial reasoning that paraphrases the instruction, and existing SFT or template methods are task-specific and generalize poorly.
Approach
Use GRPO with a rule-centric reward (format reward plus per-constraint accuracy reward checked by code or an LLM judge), a replay buffer that keeps only samples where reasoning beats reasoning-free rollouts (superior CoT enforcement), and an SFT behavior-cloning term on DeepSeek-R1 expert responses to limit policy drift.
Key results
- Qwen2.5-1.5B-Instruct average rises from 50.61 to 62.35 (+11.74%) across seven benchmarks, which the authors say is comparable to an 8B model.
- On ComplexBench with Qwen2.5-7B-Instruct, the method reaches 77.40 overall versus 74.47 for I/O and 76.26 for the best SOTA baseline (Oracle Decomposition).
- On OOD IFBench, Qwen2.5-7B-Instruct improves from 30.78 to 32.07 average, while vanilla CoT gives no gain (30.75).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RAIF training instead of I/O prompting | Qwen2.5-1.5B-Instruct · Seven instruction benchmarks (Avg.) | average score: 50.61 -> 62.35 | Helped |
| Add vanilla CoT prompting instead of I/O | Qwen2.5-1.5B-Instruct · Seven instruction benchmarks (Avg.) | average score: 50.61 -> 38.81 | Hurt |
| Add RAIF training instead of I/O prompting | Qwen2.5-7B-Instruct · Seven instruction benchmarks (Avg.) | average score: 72.58 -> 73.44 | Helped |
| Add RAIF training instead of I/O prompting | Ministral-8B-Instruct · Seven instruction benchmarks (Avg.) | average score: 68.58 -> 68.68 | No clear effect |
| Add RAIF instead of I/O prompting | Qwen2.5-7B-Instruct · ComplexBench overall | overall DRFR: 74.47 -> 77.40 | Helped |
| Add maths data to training (ablation) | Qwen2.5-7B-Instruct with CoT inference · Seven instruction benchmarks (Avg.) | average score: 68.28 -> 72.12 | Helped |
| Add complex-instruction data instead of using it alone without maths, SupCoT, or BC (ablation) | Qwen2.5-7B-Instruct with CoT inference · Seven instruction benchmarks (Avg.) | average score: 72.12 -> 73.44 | Mixed |
| Increase maths-data ratio from 1 to 5 while keeping complex ratio 1 (ablation) | Qwen2.5-7B-Instruct with CoT inference · Seven instruction benchmarks (Avg.) | average score: 73.44 -> 73.18 | Hurt |
| Add superior CoT enforcement (ablation) | Qwen2.5-7B-Instruct with maths and complex data · Seven instruction benchmarks (Avg.) | average score: 67.87 -> 71.15 | Helped |
| Add behavior cloning (ablation) | Qwen2.5-7B-Instruct with maths and complex data · Seven instruction benchmarks (Avg.) | average score: 67.87 -> 72.05 | Helped |
Surprising findings
- Vanilla CoT prompting degrades average performance for most instructed models (e.g., Qwen2.5-1.5B-Instruct 50.61 to 38.81), unlike on maths.
- SFT on distilled reasoning hurts OOD generalization on IFBench for Qwen2.5-7B-Instruct and Ministral-8B-Instruct.
Limitations
- Gains are smaller for 7B-scale models, attributed to limited dataset size.
- Ministral-8B shows limited improvement over its base.
- Semantic constraints rely on an LLM judge, adding evaluation dependence.
Experimental setup
Datasets: WildChat Alpaca DeepScaleR IFEval taxonomy constraints
Benchmarks: IFEval CELLO CFBench ComplexBench FB-Bench FollowBench InfoBench IFBench
Metrics: prompt-level loose accuracy DRFR CSR/ISR/PSR HSR/SSR/CSL average score
Baselines: I/O prompting CoT prompting Self-DeepClaude (SDC) SFT Oracle Decomposition Self-Consistency Conifer FollowComplex Self-Refine
Builds on: GRPO PPO self-instruct DeepScaleR IFEval ComplexBench
Closest papers by meaning
- When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs (NeurIPS 2025 · similarity 0.93)
- Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following (AAAI 2026 · similarity 0.91)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.91)
- Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.91)
- Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (NeurIPS 2025 · similarity 0.91)
- When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.