Research map / Multi-agent systems and games / LLM agents
MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MACM, a three-agent (Thinker, Judge, Executor) prompting method that abstracts problem conditions and objectives and iteratively mines verified new conditions, improving GPT-4 Turbo on MATH and GPT-4 on the 24-point game.
Problem
LLMs such as GPT-4 struggle with complex multi-step mathematical reasoning, and existing prompting methods (ToT, GoT) need task-specific prompts that limit generalizability.
Approach
Abstract each problem into a condition list and an objective, then use a Thinker agent to mine new conditions, a Judge agent to verify them and check sufficiency, and an Executor agent to compute the answer, with one generic prompt set across problem types.
Key results
- On MATH level 5 problems, GPT-4 Turbo accuracy rose from 54.68% to 76.73% with MACM.
- On MATH overall, MACM reached 87.92% vs 80.12% for SC-CoT (v=5) and 72.78% for I-O with GPT-4 Turbo.
- On the 24-point game, GPT-4 + MACM reached 91% vs 74% for GPT-4 + ToT (b=5); GPT-3.5 + MACM reached 67%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adding MACM multi-agent condition mining (Thinker/Judge/Executor) to GPT-4 Turbo | GPT-4 Turbo · MATH full test subset | accuracy (%): 72.78 -> 87.92 overall vs I-O | Helped |
| Adding MACM to GPT-4 Turbo | GPT-4 Turbo · MATH Level 5 | accuracy (%): 54.68 -> 76.73 | Helped |
| Adding MACM instead of SC-CoT (v=5) to GPT-4 Turbo | GPT-4 Turbo · MATH full test subset | accuracy (%): 80.12 -> 87.92 | Helped |
| Adding MACM to GPT-4 | GPT-4 · 24-point game | success rate (%): 74 -> 91 over ToT (b=5) | Helped |
| Adding MACM to GPT-3.5 (vs GPT-4 with ToT b=1) | GPT-3.5 · 24-point game | success rate (%): 67 vs 45 | Helped |
| Adding MACM to GPT-3.5 on sequence sorting | GPT-3.5 · Sequence sorting (64 elements) | accuracy (%): 89.06 -> 92 (values from GoT graph estimate) | Helped |
| Adding MACM to GPT-4 Turbo on geometry (Level 5 and full MATH) | GPT-4 Turbo · MATH geometry | accuracy (%): geometry gain 17.63 points but final accuracy only 62.74 | Mixed |
| Adding a Judge that verifies new conditions before they enter the condition list (Judge rejects incorrect conditions) (ablation) | MACM on MATH algebra example · MATH algebra case study | qualitative correctness of condition list | Helped |
| Using MACM rather than CoT/SC-CoT on the accuracy vs number-of-responses trade-off | GPT-4 Turbo on 200 initially wrong MATH questions · MATH error correction vs query budget | fraction of original errors corrected: MACM corrects nearly 90% of errors at high query counts vs about 20% for I-O, CoT, SC-CoT | Helped |
| Replacing ToT's task-specific evaluation prompt with MACM's generalized objective-based prompt (ablation) | 24-point game evaluation step · 24-point game | qualitative generalizability | No clear effect |
Surprising findings
- GPT-3.5 with MACM (67%) outperforms GPT-4 with ToT at breadth 1 (45%) on the 24-point game.
- Geometry remained the weakest category, reaching only 62.74% overall with MACM.
Limitations
- Multiple LLM invocations increase inference cost and solution time.
- Geometry problems remain difficult because the model misunderstands relationships between figures.
- Evaluation uses subsets of MATH (one-third overall, half of level 5 per category) and 100 games/sequences, with a 5-iteration cap.
Experimental setup
Datasets: MATH 24-point game (4nums.com, games 901-1000) random sequences of 64 elements
Benchmarks: MATH 24-point game sequence sorting
Metrics: accuracy success rate
Baselines: I-O CoT SC-CoT ToT GoT
Builds on: Chain of Thought Self-consistency CoT Tree of Thought Graph of Thought
Closest papers by meaning
- MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem (NeurIPS 2025 · similarity 0.87)
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving (ICLR 2024 · similarity 0.86)
- RMath: A Logic Reasoning-Focused Datasets Toward Mathematical Multistep Reasoning Tasks (AAAI 2025 · similarity 0.86)
- Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models (ICLR 2024 · similarity 0.86)
- Neuro-Symbolic Data Generation for Math Reasoning (NeurIPS 2024 · similarity 0.86)
- Building Math Agents with Multi-Turn Iterative Preference Learning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.