Research map / Large language models / Agents, code and math
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SyReLM, where a frozen small LM with a LoRA adapter translates arithmetic word problems into formal-language programs executed by a symbolic solver, trained with PPO using reasoning-specific rewards.
Problem
Small LMs lack multi-step arithmetic reasoning ability, which is often assumed to emerge only in very large LLMs; tool-augmented LLMs keep the tool detached from LM learning.
Approach
Treat arithmetic word problems as formalize-then-solve: a frozen small LM with a LoRA adapter generates pseudocode or Python, a symbolic solver evaluates it, and PPO fine-tunes the adapter using rewards from the non-differentiable solver (compilation, variable count, operator match, answer correctness).
Key results
- SyReLM with GPT-J 6B reaches 40.1% on SVAMP, a +30.65 point gain over GPT-J k-shot baseline (per text), and 53.16% on MultiArith.
- SyReLM with Vicuna 13B reaches 56.65% on SVAMP, 59% on MultiArith and 35.2% on GSM8K, outperforming PAL (53.7, 49.4, 27.5) and ART.
- Vicuna 13B with SyReLM achieves 56.65% on SVAMP, close to GPT-3.5 8-shot at 64.8%, and exceeds GPT-3.5 on GSM8K (35.2 vs 15.1) and MultiArith (59 vs 34.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add symbolic-solver FL generation and PPO training | Vicuna 13B · SVAMP | accuracy (%): 37.5 -> 56.65 | Helped |
| add symbolic-solver FL generation and PPO training | GPT-J · SVAMP | accuracy (%): 9.45 -> 40.1 | Helped |
| add reward R1 (successful compilation) (ablation) | SyReLM with Vicuna 13B · SVAMP, MultiArith, GSM8K | accuracy (%): 51.0 -> 56.65; 52.3 -> 59; 34.95 -> 35.2 | Helped |
| add reward R1 (successful compilation) (ablation) | SyReLM with GPT-J · SVAMP, MultiArith, GSM8K | accuracy (%): 39.09 -> 40.1; 57.66 -> 53.16; 3.203 -> 3.50 | Mixed |
| add reward R2 (variable-count match) (ablation) | SyReLM with Vicuna 13B · SVAMP, MultiArith, GSM8K | accuracy (%): 49.25 -> 56.65; 52.0 -> 59; 25.75 -> 35.2 | Helped |
| add reward R2 (variable-count match) (ablation) | SyReLM with GPT-J · SVAMP, MultiArith, GSM8K | accuracy (%): 39.78 -> 40.1; 54.16 -> 53.16; 3.43 -> 3.50 | Mixed |
| add reward R3 (arithmetic-operator match) (ablation) | SyReLM with Vicuna 13B · SVAMP, MultiArith, GSM8K | accuracy (%): 46.75 -> 56.65; 50.1 -> 59; 26.15 -> 35.2 | Helped |
| add reward R3 (arithmetic-operator match) (ablation) | SyReLM with GPT-J · SVAMP, MultiArith, GSM8K | accuracy (%): 35.69 -> 40.1; 47.5 -> 53.16; 3.02 -> 3.50 | Helped |
| add reward R4 (answer correctness) (ablation) | SyReLM with Vicuna 13B · SVAMP, MultiArith, GSM8K | accuracy (%): 31.08 -> 56.65; 45.8 -> 59; 22.2 -> 35.2 | Helped |
| add reward R4 (answer correctness) (ablation) | SyReLM with GPT-J · SVAMP, MultiArith, GSM8K | accuracy (%): 36.92 -> 40.1; 52 -> 53.16; 3.81 -> 3.50 | Mixed |
Surprising findings
- GPT-J with SyReLM gets only 3.50% on GSM8K, and the gain over the base model is meagre, suggesting pseudocode is ill-suited for harder problems.
- ART with GPT-J performs near random across all datasets, indicating tool-use prompting presupposes strong reasoning absent in small models.
Limitations
- Gains on GSM8K are small for GPT-J because pseudocode becomes long and complex.
- LoRA finetuning and PPO require extra GPU-hours compared with zero-shot API-based LLMs.
- Frugal LMs have limited language generation and abstract reasoning, leaving shortcomings after optimization.
Experimental setup
Datasets: ASDiv MAWPS Math23k SVAMP GSM8K MultiArith
Benchmarks: SVAMP MultiArith GSM8K
Metrics: accuracy
Baselines: PAL ART Toolformer TRICE LMFT k-shot prompting GPT-3.5 8-shot
Builds on: LoRA PPO Toolformer PAL Program-of-Thoughts prompting
Closest papers by meaning
- AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking (ICLR 2026 · similarity 0.90)
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.90)
- Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.90)
- Neuro-Symbolic Data Generation for Math Reasoning (NeurIPS 2024 · similarity 0.90)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.89)
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.