Research map / Large language models / Reasoning and chain of thought
Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Enhances small open-source LLMs' math reasoning in Hindi and English via curriculum learning (easy→medium SFT), a structured solution format, a place-value Decomposition Strategy, and bilingual combined training, releasing the IndiMathQA dataset.
Problem
Open-source LLMs of about 7B parameters perform poorly on mathematical reasoning, especially in Hindi, and lag behind closed models such as GPT-4 and Gemini Pro.
Approach
Combines curriculum-ordered SFT (easy then medium), a six-phase structured solution format, a place-value decomposition of multiplication and division in HAWP, and training on paired English and Hindi math QA data, applied to several 7B open models.
Key results
- WizardMath-7B with curriculum SFT reaches 80% on GSM8K and 45% on MATH, above the base WizardMath scores of 71% and 36%.
- Per the paper's claim, WizardMath-7B exceeds Gemini 1.0 Pro by about 5% on English datasets while lagging it by about 3% on Medium and Hard Hindi problems.
- Bilingual combined training improves Hindi benchmark accuracy over single-language fine-tuning for all models tested, per the authors' tables.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Curriculum learning: SFT on easy then easy+medium problems (ablation) | WizardMath-7B · English benchmarks (GSM8K, MATH, EMKB) | accuracy: GSM8K 71% -> 80%; MATH 36% -> 45% | Helped |
| Curriculum learning: SFT on easy then easy+medium problems (ablation) | WizardMath-7B · Hindi benchmarks (Enhanced HAWP, HMKB) | accuracy: Enhanced HAWP 68% -> 77% | Helped |
| Bilingual combined training (English+Hindi) with curriculum learning (ablation) | open-source SLLMs · HMKB Hindi benchmarks | accuracy: WizardMath-7B HMKB Medium 46% (Hindi-only) -> 57% (bilingual) | Helped |
| Decomposition Strategy (split multiplicand/dividend into place-value parts) added to HAWP fine-tuning (ablation) | LLeMMA-7B · Enhanced HAWP division | accuracy: division 0.67 -> 0.83 (HAWP+Decomposition vs Enhanced HAWP) | Helped |
| Decomposition Strategy added to HAWP fine-tuning (ablation) | OpenHathi-7B · Enhanced HAWP multiplication | accuracy: 0.22 -> 0.22 (multiplication); division 0.50 -> 0.67 | No clear effect |
| Fine-tuning on Enhanced HAWP (no decomposition) (ablation) | LLeMMA-7B · Enhanced HAWP arithmetic | accuracy: performance declined after fine-tuning, attributed to hallucinations | Hurt |
| Structured solution format (data identification, analysis, theory, methodology, computation, answer) | WizardMath-7B · Error analysis example (single probability question) | qualitative correctness: base answer 100% -> SFT answers 1/2 then 1/3 (correct) | Helped |
| Hindi-only vs English-only curriculum training gap: WizardMath-7B vs Gemini 1.0 Pro on Hindi | WizardMath-7B · Hindi benchmarks | accuracy: comparable to Gemini on Enhanced HAWP (77% vs 81%), still lags on Medium/Hard | No clear effect |
| Curriculum learning on WizardMath-7B vs Gemini 1.0 Pro on English | WizardMath-7B · English benchmarks | accuracy: GSM8K 80% vs Gemini 75%; MATH 45% vs 39% | Helped |
Surprising findings
- Fine-tuning on Hindi arithmetic data degraded LLeMMA-7B, which the authors attribute to hallucinations on new knowledge; the decomposition strategy partly recovered its performance.
- GPT-4 was tested and outperformed all fine-tuned open models by a large margin, even after the proposed curriculum and bilingual training.
Limitations
- Romanized Hindi input is not evaluated.
- Easy/Medium/Hard labelling was done manually and may be biased.
- Decomposition strategy is validated only on the single-operation HAWP dataset.
Experimental setup
Datasets: GSM8K MATH HAWP IndiMathQA PRM800K
Benchmarks: GSM8K MATH PRM800K EMKB Enhanced HAWP HMKB
Metrics: accuracy
Baselines: GPT-4 Gemini 1.0 Pro LLaMA-2-7B Mistral-7B MAmmoTH-7B LLeMMA-7B OpenHathi-7B Airavata Hi-NOLIN
Builds on: curriculum learning chain-of-thought Step-back prompting
Closest papers by meaning
- PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts (NeurIPS 2025 · similarity 0.91)
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.90)
- OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data (ICLR 2025 · similarity 0.90)
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (ICLR 2024 · similarity 0.89)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.89)
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.