Research map / Large language models / Reasoning and chain of thought

Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies rule-based R1-style GRPO with a binary format-plus-tool-call reward to train Tool-N1 tool-calling reasoning models (7B/14B) from Qwen2.5-Instruct, without curated reasoning traces.

Problem

Tool-calling LLMs are typically trained by SFT on synthetic trajectories without reasoning supervision, which yields imitative, poorly generalizing behavior; the paper asks whether rule-based RL can train reasoning tool callers and how the RL pipeline should be designed.

Approach

Use GRPO with a binary reward (1 only if the output has correct <think>/<tool_call> format and the tool call exactly matches the ground truth in name and argument key-value pairs) on existing SFT tool-calling data, so reasoning emerges without reasoning annotations; the study also compares SFT, RL, and SFT-then-RL recipes.

Key results

• Tool-N1-14B achieves 85.97% overall on BFCL, about 2% above GPT-4o (83.97%), and surpasses GPT-4o by 5.03% on API-Bank/ACEBench per the text.
• Tool-N1-7B reaches 84.82% overall on BFCL, outperforming Hammer2.1-7B (81.88%) and GPT-4o.
• On the 5,518-instance ToolACE study, pure RL (83.24%) matches or slightly exceeds Reason-SFT+RL (83.17%) and Reason-SFT (82.71%).

Surprising findings

• SFT-then-RL does not outperform pure RL under equal data budgets in the tool-calling setting.
• Removing the reasoning format constraint from the binary reward drops Live accuracy from 80.38% to 76.24%.

Limitations

• Evaluation restricted to single-turn tool-calling; multi-turn BFCL cases excluded.
• Gains on the small Live parallel categories are mixed and rely on few samples.
• Smaller models (0.5B, 1.5B) show limited improvement from post-training.

Experimental setup

Datasets: xLAM ToolACE APIGen

Benchmarks: BFCL API-Bank ACEBench

Metrics: accuracy

Baselines: GPT-4o GPT-4o-mini Gemini-2.0-Flash ToolACE-8B xLAM-2 Hammer2.1-7B Qwen2.5-7B-Instruct DeepSeek-R1 Llama3.1-70B-Instruct SFT on ToolACE/xLAM

Builds on: GRPO DeepSeek-R1 veRL APIGen

Closest papers by meaning

• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.91)
• ToolRL: Reward is All Tool Learning Needs (NeurIPS 2025 · similarity 0.91)
• ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
• Joint Optimization of Tool Creation and Use for Large Language Model Agents (NeurIPS 2026 · similarity 0.90)
• Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning (ICML 2026 · similarity 0.90)
• Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.90)