Research map / Large language models / Agents, code and math
MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MM-Bench (111 MCM/ICM problems) and MM-Agent, a four-stage LLM agent framework with hierarchical method retrieval and actor-critic modeling, evaluated against LLM agent baselines and human teams.
Problem
Current LLM agents cannot perform open-ended real-world mathematical modeling, including problem analysis, assumption-driven formulation, and end-to-end solution reporting, which goes beyond well-defined math reasoning.
Approach
An expert-inspired four-stage agent (problem analysis, hierarchical actor-critic modeling with a tri-level Hierarchical Mathematical Modeling Library retrieval, MLE-Solver-based computation, and LaTeX report generation) that decomposes modeling into subtasks with dependency graphs.
Key results
- MM-Agent with DeepSeek-R1-671B achieves overall 8.92 on 2025 problems vs 8.02 for the Human Team and 7.79 for the best baseline ResearchAgent.
- MM-Agent with GPT-4o reaches overall 8.85 on 2021-2024 problems, versus 7.91 for Human Team, and costs $0.88 per task (about 906 s runtime).
- On OPTIBENCH zero-shot, MM-Agent with GPT-4o achieves 68.8% overall accuracy vs 66.8% for GPT-4o alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MM-Agent instead of the strongest reported baseline, Human Team | GPT-4o-based modeling on MCM/ICM 2021–2024 · MCM/ICM 2021–2024 | Overall: 7.91 -> 8.85 | Helped |
| Use MM-Agent instead of the strongest reported baseline, Human Team | GPT-4o-based modeling on MCM/ICM 2025 · MCM/ICM 2025 | Overall: 8.02 -> 8.38 | Helped |
| Use MM-Agent instead of the strongest reported baseline, Human Team | DeepSeek-R1-671B-based modeling on MCM/ICM 2021–2024 · MCM/ICM 2021–2024 | Overall: 7.91 -> 8.85 | Helped |
| Use MM-Agent instead of the strongest reported baseline, Human Team | DeepSeek-R1-671B-based modeling on MCM/ICM 2025 · MCM/ICM 2025 | Overall: 8.02 -> 8.92 | Helped |
| Use MM-Agent instead of ResearchAgent | Qwen2.5-72B modeling · MCM/ICM 2021–2025 | Overall: 7.87 -> 8.37 | Helped |
| Use MM-Agent instead of GPT-4o | Zero-shot OPTIBENCH optimization · OPTIBENCH linear, w/o Table | Accuracy: 77.5% -> 79.5% | Helped |
| Use MM-Agent instead of GPT-4o | Zero-shot OPTIBENCH optimization · OPTIBENCH linear, w/ Table | Accuracy: 68.8% -> 70.0% | Helped |
| Use MM-Agent instead of GPT-4o | Zero-shot OPTIBENCH optimization · OPTIBENCH nonlinear, w/o Table | Accuracy: 48.1% -> 50.4% | Helped |
| Use MM-Agent instead of GPT-4o | Zero-shot OPTIBENCH optimization · OPTIBENCH nonlinear, w/ Table | Accuracy: 40.0% -> 42.0% | Helped |
| Replace task dependency analysis with a naive task parser (ablation) | MM-Agent · MM-Bench | Modeling Rigorousness | Hurt |
| Replace HMML with a flat retrieval library (ablation) | MM-Agent · MM-Bench | Analysis Evaluation | Hurt |
| Replace HMML with a flat retrieval library (ablation) | MM-Agent · MM-Bench | Result and Bias Analysis | Hurt |
Surprising findings
- Human-human annotator agreement on Modeling Rigorousness (0.4813) is lower than model-human agreement (0.7130), indicating annotator subjectivity.
- Agent Laboratory underperforms simpler agents like DS-Agent despite much higher token consumption (746k tokens with GPT-4o).
Limitations
- Evaluation relies largely on GPT-4o-based automatic scoring and a small number of human experts, with notable annotator disagreement on some dimensions.
- Benchmark may face contamination from publicly available competition solutions; mitigated only by using 2025 problems.
- Qwen2.5-72B could not run the Agent Laboratory workflow reliably due to instruction-following demands.
Experimental setup
Datasets: MCM/ICM problems 2000-2025 OPTIBENCH
Benchmarks: MM-Bench OPTIBENCH
Metrics: Analysis Evaluation score Modeling Rigorousness score Practicality and Scientificity score Result and Bias Analysis score Overall score (1-10) accuracy token cost runtime
Baselines: GPT-4o direct DeepSeek-R1 direct DS-Agent ResearchAgent Agent Laboratory Human Team
Builds on: MLE-Solver DS-Agent Agent Laboratory ResearchAgent
Closest papers by meaning
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.88)
- EngiAgent: Fully Connected Coordination of LLM Agents for Solving Open-ended Engineering Problems with Feasible Solutions (ICML 2026 · similarity 0.87)
- Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration (ICLR 2026 · similarity 0.87)
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.87)
- AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.87)
- THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.