Research map / Large language models / Agents, code and math

LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

ICLR 2025 · Large language models › Agents, code and math · approach: LLMs for optimization and heuristics · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLMOPT, which fine-tunes Qwen1.5-14B with a five-element optimization formulation, multi-instruction SFT, KTO alignment, and execution-feedback self-correction to formulate and solve diverse optimization problems from natural language.

Problem

Automating the formulation and solver-code generation for optimization problems described in natural language suffers from low accuracy and limited generality across optimization problem types.

Approach

A universal five-element formulation (Sets, Parameters, Variables, Objective, Constraints) as an intermediate representation, trained via multi-instruction SFT and KTO alignment on expert- and GPT-4-labeled data, plus an auto-testing loop that self-corrects based on solver execution output.

Key results

• LLMOPT (Qwen1.5-14B) reaches 93.0% solving accuracy on NL4Opt, 97.0% on Mamo Easy, 68.0% on Mamo Complex, and 46.0% on IndustryOR, versus 86.5%, 82.3%, 37.9%, 38.0% for the best ORLM variants.
• On prompt-based comparisons, LLMOPT reaches 83.8% on NLP4LP and 72.7% on ComplexOR versus OptiMUS's 72.0% and 66.7%.
• Authors report an average 11.08% solving accuracy improvement over state-of-the-art across six datasets; removing self-correction drops NL4Opt SA from 93.0% to 57.0%.

Measured effects reported in the paper

Surprising findings

• Five-element formulation can slightly lower execution rate even while improving solving accuracy, as the LLM otherwise oversimplifies problems into error-free but inaccurate code.
• Fine-tuning on optimization data showed no significant seesaw degradation on 10 general tasks (average +0.3%).

Limitations

• Relies on expert-labeled and GPT-4-labeled data, which is scarce and time-consuming to produce.
• Does not address large-scale problems whose data reside in databases or files rather than natural language.
• Several comparisons cite published numbers from prior papers rather than re-running them, and the o1 evaluation is limited to 20 problems.

Experimental setup

Datasets: NL4Opt Mamo Easy Mamo Complex IndustryOR NLP4LP ComplexOR UltraFeedback (KTO, binarized) Tulu-style instruction data (AllenAI)

Benchmarks: NL4Opt Mamo Easy Mamo Complex IndustryOR NLP4LP ComplexOR

Metrics: solving accuracy execution rate average solving times

Baselines: GPT-4 GPT-4o GPT-4-turbo ORLM Reflexion Chain-of-Experts OptiMUS Qwen2-72B

Builds on: KTO ORLM Chain-of-Experts OptiMUS Reflexion

Closest papers by meaning

• Autoformulation of Mathematical Optimization Models Using LLMs (ICML 2025 · similarity 0.90)
• OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling (ICLR 2025 · similarity 0.90)
• OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling (ICML 2025 · similarity 0.89)
• SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Search (NeurIPS 2025 · similarity 0.89)
• Large Language Models as End-to-end Combinatorial Optimization Solvers (NeurIPS 2025 · similarity 0.89)
• MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling (NeurIPS 2026 · similarity 0.89)