Research map / Large language models / Agents, code and math
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains LLaMA-2 and CodeLLaMA models (7B to 70B) to solve math problems by interleaving natural-language rationale with executed program calls, using GPT-4-generated trajectories for imitation learning plus an output space shaping step.
Problem
Open-source LLMs struggle with complex mathematical reasoning; natural-language CoT is weak at precise computation while program-only (PAL) methods lack planning and error handling.
Approach
Interleave natural-language rationales with program-based tool calls in multi-round trajectories, train via imitation learning on GPT-4-annotated trajectories, then apply output space shaping (sampling valid trajectories and correcting invalid ones with a 34B teacher) to broaden the model's output space.
Key results
- ToRA-Code-34B reaches 50.8% on MATH, beating GPT-4 CoT (42.5%) and approaching GPT-4 PAL (51.8%).
- ToRA-7B reaches 44.6% on MATH (ToRA-Code variant), surpassing WizardMath-70B (22.7%) by about 22 points.
- Output space shaping improves average accuracy by 3.4% on GSM8k and 4.0% on MATH; ToRA-70B goes from 47.3% to 49.7% on MATH.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CodeLLaMA-34B with ToRA tool-integrated reasoning and output space shaping | CodeLLaMA-34B · MATH | accuracy: 23.9 -> 50.8 | Helped |
| Continue code training to make ToRA-Code rather than ToRA | ToRA, same model size · MATH | accuracy: 40.1 -> 44.6; 43.0 -> 48.1 | Helped |
| Remove output space shaping (ablation) | ToRA-Code-7B · MATH overall | accuracy: 44.6~ -> 40.2~ | Hurt |
| Add output space shaping (ablation) | ToRA-Code-7B · MATH overall | accuracy: 40.2~ -> 44.6~ | Helped |
| Remove rationale from the interleaved format (ablation) | ToRA-Code-7B · MATH overall | accuracy: 44.6~ -> 36.8~ | Hurt |
| Add rationale to program-based tool use (ablation) | ToRA-Code-7B · MATH overall | accuracy: 36.8~ -> 44.6~ | Helped |
| Remove output space shaping (ablation) | ToRA-Code-13B · MATH overall | accuracy: 48.1~ -> 44.6~ | Hurt |
| Add output space shaping (ablation) | ToRA-Code-13B · MATH overall | accuracy: 44.6~ -> 48.1~ | Helped |
| Remove rationale from the interleaved format (ablation) | ToRA-Code-13B · MATH overall | accuracy: 48.1~ -> 40.2~ | Hurt |
| Add rationale to program-based tool use (ablation) | ToRA-Code-13B · MATH overall | accuracy: 40.2~ -> 48.1~ | Helped |
| Remove output space shaping (ablation) | ToRA-Code-34B · MATH overall | accuracy: 50.8 -> 47.4~ | Hurt |
| Add output space shaping (ablation) | ToRA-Code-34B · MATH overall | accuracy: 47.4~ -> 50.8 | Helped |
Surprising findings
- Rationale-based fine-tuning (WizardMath-70B) underperforms its base model on TabMWP (49.8% vs 57.5%), while tool-integrated models generalize better.
- Feeding GPT-4 human rationales as hints caused it to copy hints and ignore tool outputs, lowering trajectory quality.
Limitations
- Failure analysis on 100 MATH trajectories: reasoning errors 38%, diagram understanding 21%, inappropriate tool use 10%.
- Weak on Level 5 MATH problems and diagram-heavy geometry/precalculus questions.
- Training data is small (about 16k GPT-4 trajectories) and relies on GPT-4 annotation.
Experimental setup
Datasets: GSM8k MATH GSM-Hard SVAMP TabMWP ASDiv SingleEq SingleOp AddSub MultiArith
Benchmarks: GSM8k MATH GSM-Hard SVAMP TabMWP ASDiv MAWPS
Metrics: accuracy
Baselines: GPT-4 CoT GPT-4 PAL ChatGPT PAL WizardMath Platypus-2 CodeLLaMA PAL LLaMA-2 RFT Toolformer
Builds on: PAL Rejection sampling fine-tuning Toolformer LLaMA-2 CodeLLaMA
Closest papers by meaning
- THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning (ICLR 2026 · similarity 0.88)
- DenseSteer: Steering Small Language Models towards Dense Math Reasoning (ICML 2026 · similarity 0.88)
- HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs (ICML 2026 · similarity 0.88)
- RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics (NeurIPS 2025 · similarity 0.88)
- We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning (ICLR 2026 · similarity 0.88)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.