Research map / Large language models / Agents, code and math
Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes decoupling LLM mathematical reasoning into field-specific (algebra, geometry, analysis, topology) and logical (conceptual understanding, forward formal reasoning, counterexample-driven backward reasoning) atomic capabilities, with datasets and an evaluation of advanced models plus LoRA-based cross-capability interaction experiments.
Problem
Current LLM math evaluation and training rely on end-to-end accuracy and scaling data, offering little insight into whether models acquire underlying mathematical concepts and reasoning atoms rather than memorizing training problems.
Approach
Decompose mathematical ability into atomic capabilities along field and logical dimensions, build separate train/test data for each, and measure how stimulating one atomic capability transfers to others via supervised LoRA fine-tuning.
Key results
- Qwen2.5-Math-7B-Instruct reaches 80.5 on low-level algebra and QwQ-32B 85.1, while OpenAI o1 reaches 93.6; topology and geometry are weaker across open models.
- On the counterexample task, Deepseek-R1 scores 80.7 F-1 and Gemini2.5-pro achieves 65.1 strict align, whereas most open-source models average around 30 F-1.
- Training Qwen2.5-Math-7B-Instruct with LoRA on algebra data raises geometry low-level accuracy from 52.1 to 65.7, larger than in-field geometry training (+6.1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on Algebra atomic-capability data (ablation) | Qwen2.5-Math-Instruct-7B · Geometry low | accuracy: 52.1 -> 65.7 | Helped |
| Train on Algebra atomic-capability data (ablation) | Qwen2.5-Math-Instruct-7B · Analysis low | accuracy: 67.7 -> 75.8 | Helped |
| Train on Algebra atomic-capability data (ablation) | Qwen2.5-Math-Instruct-7B · Algebra low | accuracy: 80.5 -> 80.2 | Hurt |
| Train on Topology atomic-capability data (ablation) | Qwen2.5-Math-Instruct-7B · Analysis high | accuracy: 66.5 -> 59.2 | Hurt |
| Train on conceptual understanding fill-in-the-blank tasks (ablation) | Qwen2.5-Math-Instruct-7B · Forward reasoning formal language | accuracy: 34.4 -> 53.5 | Helped |
| Train on conceptual understanding fill-in-the-blank tasks (ablation) | Qwen2.5-Math-Instruct-7B · Backward reasoning statement judgment | F-1: 30.2 -> 40.1 | Helped |
| Train on forward reasoning data (ablation) | Qwen2.5-Math-Instruct-7B · Concept attribute description | accuracy: 34.4 -> 25.1 | Hurt |
| Train on backward reasoning data (ablation) | Qwen2.5-Math-Instruct-7B · Forward reasoning formal language | accuracy: 34.4 -> 50.3 | Helped |
| Train on high-difficulty field data (ablation) | Qwen2.5-Math-Instruct-7B · Algebra high | accuracy: 65.2 -> 68.3 | Helped |
| Train on low-difficulty field data (ablation) | Qwen2.5-Math-Instruct-7B · Analysis high | accuracy: 66.5 -> 62.2 | Hurt |
| Train on conceptual understanding tasks (ablation) | Qwen2.5-Math-Instruct-7B · Analysis high | accuracy: 66.5 -> 72.3 | Helped |
Surprising findings
- Training on algebra data yields larger gains in geometry and analysis than training on the target field itself.
- Training solely on conceptual definition completion improves forward reasoning by 19.1 points and counterexample F-1 by 9.9 points on Qwen2.5-Math-7B-Instruct.
Limitations
- Authors state they did not explore advanced strategies such as curriculum learning or reinforcement learning for stimulating atomic capabilities.
- No error bars or statistical significance reported in main experiments; only a limited appendix with multiple seeds.
- Human evaluation covers only 20 sampled problems per dataset for two models.
Experimental setup
Datasets: MATH GSM8K Gaokao-Bench OlympiadBench AIME MMLU DeepMath NaturalProofs LeanWorkbook CounterMATH
Benchmarks: Atomic field capability dataset (algebra, geometry, analysis, topology; levels 1-2) Atomic logic capability dataset (attribute description, definition, forward formal reasoning, counterexample) CounterMATH metrics
Metrics: accuracy F-1 Example (%) Strict align (%) Loose align (%)
Baselines: Deepseek-Math-7B-RL Eurus-2-7B-PRIME Qwen2.5-Math-7B-Instruct NuminaMath-7B-TIR InternLM2-Math-Plus-7B/20B Abel-7B/13B WizardMath-7B MetaMath-Mistral-7B Xwin-Math-7B/13B QwQ-32B GPT-4o OpenAI o1
Builds on: CounterMATH chain-of-thought LoRA NaturalProofs
Closest papers by meaning
- CogMath: Assessing LLMs' Authentic Mathematical Ability from a Human Cognitive Perspective (ICML 2025 · similarity 0.91)
- OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization (NeurIPS 2025 · similarity 0.90)
- One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMs (ICML 2025 · similarity 0.89)
- RMath: A Logic Reasoning-Focused Datasets Toward Mathematical Multistep Reasoning Tasks (AAAI 2025 · similarity 0.89)
- UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.89)
- Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problems (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.