Research map / Large language models / Reasoning and chain of thought

Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cascade RL, sequential domain-wise RL (RLHF, instruction following, math, code, SWE) after SFT to build Nemotron-Cascade 8B unified and 14B thinking reasoning models from Qwen3 base models.

Problem

RL for general-purpose reasoning models suffers from cross-domain heterogeneity in response length and verification latency, complicating infrastructure, curriculum and hyperparameter choice; unified thinking/non-thinking models also lag dedicated thinking models.

Approach

Train RL sequentially across domains (RLHF, instruction-following RL, math RL, code RL, SWE RL) rather than blending all domains, tailoring hyperparameters and length curricula per domain; RL is argued to resist catastrophic forgetting. Unified model trained with parallel thinking/non-thinking SFT data and equal per-mode prompt splits in RLHF.

Key results

• Nemotron-Cascade-14B-Thinking reaches 77.5 on LCB v5 and 74.6 on LCB v6, surpassing DeepSeek-R1-0528 (74.8 and 73.3) under 64K budget.
• Nemotron-Cascade-8B (unified) reaches 74.3 on LCB v5 and 71.1 on LCB v6, comparable to DeepSeek-R1-0528 (671B).
• Nemotron-Cascade-14B-Thinking achieves AIME 2024 89.7 and AIME 2025 83.3; 8B reaches 89.5 and 80.1 without tools.

Surprising findings

• RLHF with a reward model improved math and code benchmarks despite containing no math or code prompts.
• Qwen3 reasoning checkpoints underperformed Qwen2.5-Instruct as reward model backbones, even in non-thinking mode.

Limitations

• Due to resource constraints, only a 14B thinking model was trained beyond the 8B unified model.
• RLHF with reward model degrades IFEval, indicating conflict between preference reward and strict instruction constraints.
• Reward model may give unreliable signals for math/code, so these prompts were excluded from RLHF.

Experimental setup

Datasets: HelpSteer2 HelpSteer3 AceReason-Nemotron-1.1 SFT prompts AceReason-Math OpenCodeReasoning OpenCoder-Stage2 Llama-Nemotron post-training data LMSYS-Chat-1M Tulu-3-SFT SWE-Bench-Train SWE-Fixer-Train SWE-reBench

Benchmarks: MMLU MMLU-Pro GPQA-Diamond ArenaHard IFEval IFBench AIME 2024 AIME 2025 LiveCodeBench v5 LiveCodeBench v6 LiveCodeBench Pro 25Q2 SWE-bench Verified

Metrics: pass@1 (avg@k)

Baselines: DeepSeek-R1-0528 Qwen3-8B Qwen3-14B Nemotron-Nano-9B-v2 Gemini-2.5-Flash-Thinking Gemini-2.5-Pro-06-05 o4-mini Qwen3-235B-A22B DeepSWE-32B

Builds on: GRPO AceReason-Nemotron DeepSeek-R1 Agentless

Closest papers by meaning

• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.91)
• AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.89)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.89)
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.89)