Research map / Large language models / Agents, code and math
Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TeLL, a template-driven, LLM-paraphrased pipeline that generates tabular math word problems and builds the TabMWP-TeLL dataset with illustrative step-by-step solutions, showing gains when fine-tuning Mistral, Qwen 2, and Llama 3.
Problem
Fine-tuning LLMs for tabular math word problem (TMWP) solving needs large, high-quality data, but existing generated MWPs lack either correctness (hallucinated answers) or diversity (template-only problems).
Approach
Extract abstract templates (question, table, answer, solution) from real TabMWP samples, augment them with an LLM to new question types, instantiate them programmatically to ensure correct answers, then use an LLM to paraphrase the problems into diverse contexts, and enrich solutions with illustrative reasoning steps.
Key results
- Llama 3-8B fine-tuned on TabMWP + TabMWP-TeLL reaches 98.07% overall accuracy on TabMWP test, up from 94.29% with TabMWP alone.
- Mistral, Qwen 2, and Llama 3 gain 3.96, 3.70, and 3.78 points respectively from adding TabMWP-TeLL.
- Human verification of 1,000 sampled TabMWP-TeLL examples found 97.5% correctness.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TabMWP-TeLL training data | Mistral 7B, Qwen 2 7B, and Llama 3 8B · TabMWP test | accuracy (Avg.): 92.66 -> 96.62; 93.59 -> 97.29; 94.29 -> 98.07 | Helped |
| Use template-based problems instead of the full TeLL generation method (ablation) | Mistral 7B, Qwen 2 7B, and Llama 3 8B · TabMWP test | accuracy (Overall): 96.62 -> 93.20; 97.29 -> 93.75; 98.07 -> 94.35 | Hurt |
| Generate questions and tables with LLMs rather than using the full TeLL method (ablation) | Mistral 7B, Qwen 2 7B, and Llama 3 8B · TabMWP test | accuracy (Overall): 96.62 -> 92.92; 97.29 -> 93.86; 98.07 -> 94.55 | Hurt |
| Use LLMs only to rewrite questions rather than the full TeLL method (ablation) | Mistral 7B, Qwen 2 7B, and Llama 3 8B · TabMWP test | accuracy (Overall): 96.62 -> 94.08; 97.29 -> 94.60; 98.07 -> 95.46 | Hurt |
| Add illustrative step-by-step solutions instead of free-form solutions (ablation) | Mistral 7B, Qwen 2 7B, and Llama 3 8B · TabMWP test | accuracy: 92.66 -> 94.29; 93.59 -> 95.86; 94.29 -> 96.21 | Helped |
| Train on TabMWP-TeLL only instead of TabMWP (ablation) | Llama 3 8B · Sampled stem-leaf plot problems | accuracy (Avg.): 82.49 -> 88.24 | Helped |
| Train on TabMWP and TabMWP-TeLL together instead of TabMWP only (ablation) | Llama 3 8B · Sampled stem-leaf plot problems | accuracy (Avg.): 82.49 -> 97.45 | Helped |
| Train on TabMWP-TeLL only instead of TabMWP (ablation) | Llama 3 8B · TabMWP test, question types 3, 6, and 8 | accuracy: 66.67 -> 83.88; 69.46 -> 73.65; 68.92 -> 83.27 | Helped |
| Train on TabMWP and TabMWP-TeLL together instead of TabMWP only (ablation) | Llama 3 8B · TabMWP test, question types 3, 6, and 8 | accuracy: 66.67 -> 94.14; 69.46 -> 96.51; 68.92 -> 95.82 | Helped |
| Increase the proportion of TabMWP-TeLL training data (ablation) | Fine-tuned LLMs · TabMWP test | accuracy | Helped |
Surprising findings
- Template-only generated problems yield only small gains (0.54 for Mistral), while the LLM-paraphrased questions alone gave more improvement than the full method for Mistral on MC questions.
Limitations
- Human verification found generation errors including incomplete paraphrased questions, incorrect paraphrased solutions, and grammatical errors.
- Evaluation is limited to the TabMWP benchmark and the question types derived from it (25 types).
Experimental setup
Datasets: TabMWP TabMWP-TeLL
Benchmarks: TabMWP
Metrics: exact match accuracy
Baselines: UnifiedQA TAPEX TaCo GPT-3 ChatGPT GPT-4 PromptPG PoT Chameleon
Builds on: TabMWP QLoRA Chain-of-Thought
Closest papers by meaning
- Accurate and Regret-Aware Numerical Problem Solver for Tabular Question Answering (AAAI 2025 · similarity 0.88)
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning (ICML 2024 · similarity 0.87)
- Augmenting Math Word Problems via Iterative Question Composing (AAAI 2025 · similarity 0.87)
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (ICLR 2024 · similarity 0.87)
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.87)
- Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.