Research map / Large language models / Agents, code and math
Mathematical Capabilities of ChatGPT
Simon Frieder, Luca Pinchetti, Alexis Chevalier, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Julius Berner
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Upgrade ChatGPT from the 9-January-2023 version to the 30-January-2023 version | ChatGPT · GHOSTS | average rating: 3.20 -> 3.29 | No clear effect |
| Use GPT-4 instead of ChatGPT | ChatGPT · miniGHOSTS | average rating: 3.20 -> 4.15 | Improved |
| Use GPT-4 instead of ChatGPT | ChatGPT · Symbolic-Integration | average rating: 2.51 -> 3.50 | Improved |
| Use GPT-4 instead of ChatGPT | ChatGPT · MATH | perfect-rating share: 29% -> 70% | Improved |
| Apply prompt engineering to complex questions | ChatGPT · GHOSTS complex questions | rating | Improved |
Closest papers by meaning
- Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification (ICLR 2024 · similarity 0.89)
- RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics (NeurIPS 2025 · similarity 0.88)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.88)
- MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code (ICLR 2025 · similarity 0.88)
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.87)
- MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
