Research map / Large language models / Instruction and fine-tuning
Interpreting and Improving Large Language Models in Arithmetic Calculation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies sparse attention heads and MLPs in LLaMA2 that carry out two-operand arithmetic via path patching, and fine-tunes only 32 key heads (precise SFT) to improve math benchmarks while preserving generic performance.
Problem
Arithmetic calculation in LLMs is poorly understood mechanistically, making reliability hard to ensure; it is unclear which internal components perform calculation and whether they can be selectively tuned to improve math ability.
Approach
Use path patching with counterfactual prompts (operators/operands obfuscated) to locate sparse calculation-related attention heads and MLPs, validate them via mean-ablation knockout and attention/embedding analysis, then fine-tune only those heads (precise SFT) instead of the full model.
Key results
- Precise SFT on 32 key heads (0.07B params) for LLaMA2-7B improves GSM8K from 14.6 to 27.4 and SVAMP from 34.7 to 55.8, versus Full SFT's 24.6 and 50.3.
- Precise SFT keeps LLaMA2-7B MMLU at 59.6 (vs 59.8 base) while Full SFT drops it to 54.0.
- Knocking out identified key heads reduces accuracy by about 70% on the synthetic addition task, while random-rank knockouts keep performance stable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · GSM8K | accuracy: 14.6 -> 27.4 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · AddSub | accuracy: 30.5 -> 50.6 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · SingleEq | accuracy: 65.4 -> 69.7 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · SVAMP | accuracy: 34.7 -> 55.8 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · MMLU | accuracy: 46.0 -> 46.4 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · CSQA | accuracy: 59.8 -> 59.6 | Hurt |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-13B unchanged | LLaMA2-13B · GSM8K | accuracy: 28.7 -> 46.3 | Helped |
| fine-tune the top 32 key attention heads instead of leaving LLaMA2-13B unchanged | LLaMA2-13B · SVAMP | accuracy: 45.7 -> 66.6 | Helped |
| fine-tune all parameters instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · MMLU | accuracy: 46.0 -> 40.5 | Hurt |
| fine-tune all parameters instead of leaving LLaMA2-7B unchanged | LLaMA2-7B · CSQA | accuracy: 59.8 -> 54.0 | Hurt |
| fine-tune top 8 heads rather than top 32 heads (ablation) | Precise SFT on LLaMA2-7B · GSM8K | accuracy: 27.4 -> 25.4 | Hurt |
| fine-tune top 16 heads rather than top 32 heads (ablation) | Precise SFT on LLaMA2-7B · GSM8K | accuracy: 27.4 -> 26.5 | Hurt |
Surprising findings
- Fewer than 5% of attention heads carry most of the calculation effect, and knocking out about 32 heads causes a large accuracy drop.
- Full-model SFT degrades generic benchmarks (about 5% on MMLU and CSQA) while tuning only key heads preserves them.
Limitations
- Experiments focus mainly on the LLaMA2 family, with limited generalization evidence to other models
- Only two-operand arithmetic is studied in depth; math word problems require additional skills such as text comprehension
- Tuning MLPs improves math further but costs efficiency and generic performance; how to tune MLPs precisely is left open
Experimental setup
Datasets: GSM8K AddSub SingleEq SVAMP MMLU CSQA MMLU-Humanities
Benchmarks: GSM8K AddSub SingleEq SVAMP MMLU CSQA
Metrics: accuracy training speed (samples/sec) tuned parameters
Baselines: Full SFT
Builds on: path patching causal mediation analysis mean ablation LLaMA2
Closest papers by meaning
- Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.92)
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.90)
- Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning (AAAI 2024 · similarity 0.89)
- Number Cookbook: Number Understanding of Language Models and How to Improve It (ICLR 2025 · similarity 0.89)
- Who Reasons in the Large Language Models? (NeurIPS 2025 · similarity 0.89)
- Controlling Equational Reasoning in Large Language Models with Prompt Interventions (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.