Research map / Large language models / Reasoning and chain of thought
MathMistake Checker: A Comprehensive Demonstration for Step-by-Step Math Problem Mistake Finding by Prompt-Guided LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MathMistake Checker, a demo system that OCRs handwritten math answers and uses prompt-guided LLMs (with PedCoT prompting) to find step-by-step mistakes without reference answers.
Problem
Automatically grading step-by-step student solutions to math word and calculation problems, especially without reference answers, from image-based handwritten inputs.
Approach
A two-stage system: a multi-phase OCR pipeline (segmentation, text recognition, layout analysis, order detection) converts student work images to text, then an LLM grader uses a regenerate-extract-compare-evaluate prompting process (PedCoT-based) to flag step-level mistakes without reference answers.
Key results
- No quantitative results are reported; the paper only describes qualitative demonstrations on calculation and word problems via a demo interface.
Limitations
- No quantitative evaluation of grading accuracy or OCR performance is reported.
- Demonstration only; effectiveness claims rest on qualitative case studies.
- Feedback quality improvements are stated as future work.
Experimental setup
Builds on: PedCoT chain-of-thought prompting Pointer Network
Closest papers by meaning
- UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.86)
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning (ICML 2024 · similarity 0.86)
- Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation (ICML 2025 · similarity 0.85)
- DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models (ICLR 2025 · similarity 0.85)
- Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems (ICLR 2025 · similarity 0.85)
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.