Research map / Large language models / Agents, code and math
OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OptiTree, a tree-search method over an automatically built modeling tree of OR problem categories that retrieves and integrates hierarchical modeling thoughts to decompose complex optimization modeling tasks into simpler subproblems.
Problem
LLM-based optimization modeling from natural language struggles on complex OR problems, where fixed-step decomposition (variables, constraints, objectives) often yields incorrect variable definitions.
Approach
Organize OR problems into a subproblem order-preserving modeling tree with statement thoughts and modeling thoughts per node; recursively search the tree for the maximal matching subproblem and synthesize global modeling thoughts from the hierarchy, updating the tree with failed problems.
Key results
- With DeepSeek-V3, OptiTree reaches 81.5% on MAMO ComplexLP, 84.2% on ComplexOR and 54.0% on IndustryOR, versus 51.6%, 79.0% and 46.0% for MCTS.
- With GPT-4o, OptiTree reaches 45.8% on OptMATH versus 37.3% for MCTS and 17.5% for standard prompting.
- Subproblems are identified for an average of 88% of problems; tree construction one-time cost is under three hours.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OptiTree instead of the strongest reported baseline | GPT-4o · MAMO ComplexLP | solving accuracy: 68.0 -> 81.0 | Helped |
| use OptiTree instead of the strongest reported baseline | DeepSeek-V3 · ComplexOR | solving accuracy: 79.0 -> 84.2 | Helped |
| use OptiTree instead of the strongest reported baseline | DeepSeek-V3 · IndustryOR | solving accuracy: 46.0 -> 54.0 | Helped |
| add statement thoughts for subproblem identification (ablation) | OptiTree with GPT-4o · MAMO ComplexLP | solving accuracy: 69.2 -> 81.0 | Helped |
| add statement thoughts for subproblem identification (ablation) | OptiTree with DeepSeek-V3 · NL4Opt | solving accuracy: 87.1 -> 98.3 | Helped |
| allow tree search depth 3 rather than 1 (ablation) | OptiTree with GPT-4o · ComplexOR | solving accuracy: 68.4 -> 79.0 | Helped |
| allow unrestricted rather than depth-3 search (ablation) | OptiTree with DeepSeek-V3 · ComplexOR | solving accuracy: 79.0 -> 84.2 | Helped |
| use tree search instead of selecting subproblems directly from a pool (ablation) | OptiTree · modeling datasets | solving accuracy | Helped |
| add modeling thoughts when modeling the identified subproblems (ablation) | OptiTree · challenging datasets | solving accuracy | Helped |
| increase search depth (ablation) | OptiTree · modeling datasets | solving accuracy | Helped |
Surprising findings
- Without statement thoughts, subproblem identification accuracy drops sharply, showing that summarizing problems into atomic statements reduces LLM hallucination in matching.
- Building the tree from only 100 OR-Instruct problems performs comparably to the full tree, suggesting data efficiency.
Limitations
- Remaining errors are dominated by missing constraints, which depend on domain knowledge.
- Performance is bounded by the reasoning ability of the base LLM, e.g. on logic-heavy staff rostering problems.
- Results are reported with prompt-based proprietary or API models; the method was not applied to fine-tuned small models except in a supplementary experiment.
Experimental setup
Datasets: OR-Instruct 3K IndustryOR NL4Opt MAMO EasyLP MAMO ComplexLP ComplexOR OptiBench OptMATH
Benchmarks: NL4Opt MAMO EasyLP MAMO ComplexLP ComplexOR IndustryOR OptiBench OptMATH
Metrics: solving accuracy code pass rate partial credit match rate time cost
Baselines: Standard CoT CoE OptiMUS MCTS DeepSeek-R1 OpenAI-o1 ORLM Evo-Step OptMATH LLMOPT
Builds on: Chain-of-Thoughts Tree-of-Thoughts Buffer-of-Thoughts ORLM
Closest papers by meaning
- Autoformulation of Mathematical Optimization Models Using LLMs (ICML 2025 · similarity 0.90)
- MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling (NeurIPS 2026 · similarity 0.88)
- OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling (ICML 2025 · similarity 0.87)
- OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling (ICLR 2025 · similarity 0.87)
- Large Language Models as End-to-end Combinatorial Optimization Solvers (NeurIPS 2025 · similarity 0.87)
- Enhancing Decision Making Through the Integration of Large Language Models and Operations Research Optimization (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.