Research map / Large language models / Agents, code and math
PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds PDE-Controller, a set of LLMs (Translator, Controller, Coder) fine-tuned from MathCoder2-DeepSeekMath-7B that autoformalize PDE control problems into STL, propose subgoal STLs via DPO, and synthesize Gurobi solver code, with a 2M-sample synthetic dataset.
Problem
LLMs lack the scientific reasoning and formalization skills needed to control PDE-governed systems from informal natural language descriptions; existing math LLM datasets and benchmarks do not cover PDE control.
Approach
Decompose PDE control into autoformalization of constraints into STL, subgoal-STL proposal treated as an intermediate reasoning step (trained with preference pairs ranked by simulated utility gain), and program synthesis into Gurobi-based solver code.
Key results
- Controller achieves heat utility gain 1.453 vs 0.897 for the second-best (GPT-4o), a 62% improvement.
- Translator reaches IoU 0.992 on synthetic heat data vs 0.772 for MathCoder2; Coder executability 0.9978 vs 0.9592.
- On manually written data, Translator IoU is 0.7108 (heat) and 0.6493 (wave), with executability above 82% for Coder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the translator and coder for PDE autoformalization and program synthesis | MathCoder2 · Heat test | IoU: 0.772 -> 0.992 | Helped |
| Fine-tune the translator and coder for PDE autoformalization and program synthesis | MathCoder2 · Wave test | Utility RMSE: 0.1089 -> 0.0076 | Helped |
| Train the Controller to propose subgoal STLs | PDE control · Heat PDE control, all | Utility Gain: 0.897 -> 1.453 | Helped |
| Train the Controller to propose subgoal STLs | PDE control · Wave PDE control, all | Utility Gain: 0.583 -> 0.658 | Helped |
| Train the Controller to propose subgoal STLs | PDE control · Wave PDE control, all | Success Rate: 0.773 -> 0.699 | Hurt |
| Use PDE-Controller instead of MathCoder2 for unseen 4-constraint STLs | Heat autoformalization · Heat test, unseen 4-constraint STLs | IoU: 0.8154 -> 0.934 | Helped |
| Use PDE-Controller instead of MathCoder2 for unseen 4-constraint STLs | Wave program synthesis · Wave test, unseen 4-constraint STLs | Executability: 0.8 -> 0.8 | No clear effect |
| Use PDE-Controller instead of MathCoder2 for unseen 4-constraint STLs | Wave program synthesis · Wave test, unseen 4-constraint STLs | Utility RMSE: 0.2393 -> 0.1515 | Helped |
| Replace MathCoder2's translator with the PDE-Controller Translator (ablation) | MathCoder2 Translator · Heat autoformalization | IoU: 0.772 -> 0.992; +28.5% | Helped |
| Replace MathCoder2's coder with the PDE-Controller Coder (ablation) | MathCoder2 Coder · Heat program synthesis | Executability: 0.9592 -> 0.9978; +4.02% | Helped |
| Replace MathCoder2's coder with the PDE-Controller Coder (ablation) | MathCoder2 Coder · Heat program synthesis | Utility RMSE: 0.2058 -> 0.0173; -91.6% | Helped |
| Use Translator-predicted STLs instead of ground-truth STLs as Coder input (ablation) | PDE-Controller Coder · Heat program synthesis | Utility RMSE: 0.0173 -> 0.0174; +0.57% | Hurt |
Surprising findings
- On manually written data, all models fail to generate meaningful subgoal reasoning steps due to invalid STL proposals.
- GPT o1-mini has low utility RMSE on wave data yet only 39% executability, so RMSE alone is misleading.
Limitations
- Only 1D heat and wave problems with linear STL constraints are studied.
- Subgoal reasoning largely fails on human-written, noisier problems.
Experimental setup
Datasets: synthetic PDE control problems (heat and wave) generated from STL templates 17 manually written heat and 17 manually written wave problems (test)
Benchmarks: synthetic heat and wave PDE control test sets manually written PDE control problems unseen 4-constraint STL formats
Metrics: IoU executability utility RMSE success rate utility gain valid STL ratio
Baselines: MathCoder2-DeepSeekMath-7B GPT-4o GPT o1-mini random sampling
Builds on: MathCoder2 direct preference optimization signal temporal logic lora
Closest papers by meaning
- Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning (AAAI 2024 · similarity 0.84)
- Controlling Equational Reasoning in Large Language Models with Prompt Interventions (AAAI 2025 · similarity 0.84)
- Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers (ICML 2025 · similarity 0.84)
- MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem (NeurIPS 2025 · similarity 0.84)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.83)
- PINNsAgent: Automated PDE Surrogation with Large Language Models (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.