Research map / Large language models / Agents, code and math
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Scientific Generative Agent (SGA), a bilevel framework where an LLM proposes discrete scientific hypotheses (equations, molecular SMILES) and a differentiable simulator optimizes continuous parameters, with exploit/explore temperature control.
Problem
Automating physical scientific discovery requires combining LLM reasoning over discrete symbolic hypotheses (physics equations, molecule structures) with simulation-grounded feedback for continuous parameters, which LLMs alone cannot simulate.
Approach
A bilevel optimization: an outer LLM-driven evolutionary search proposes discrete expressions and continuous parameterizations, while an inner gradient-based loop fits continuous parameters through differentiable simulation; the outer loop uses a top-k heap with a 1:3 exploit/explore temperature split (0.5 vs 1.0).
Key results
- On constitutive law task (a), SGA reaches loss 5.2e-5 versus 128.0 for the best LLM baseline (Eureka); on molecule task (f) it reaches 0.11 versus 7.1 for FunSearch.
- On SRBench-style symbolic regression, SGA achieves R2 0.9990 and MSE 1.7e+4, beating Operon (R2 0.9988).
- Removing bilevel optimization degrades performance substantially, yet still beats LLM-only baselines on most tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add bilevel optimization (simulation-based continuous optimization) (ablation) | SGA · Constitutive law search (a–d) and molecule design (e–h) | loss: 90.2 -> 5.2e-5 (a); 517.0 -> 2.1e-1 (b); 83.6 -> 6.0e-2 (c); 68.4 -> 1.4e-12 (d); 8.6e-1 -> 1.3e-4 (e); 9.1 -> 1.1e-1 (f | Helped |
| Add exploitation alongside exploration (ablation) | SGA with bilevel optimization · Constitutive law search (a–c) and molecule design (e–g) | loss: 3.0e-3 -> 5.2e-5 (a); 3.9e-1 -> 2.1e-1 (b); 6.6e-2 -> 6.0e-2 (c); 4.0e-4 -> 1.3e-4 (e); 1.5e-1 -> 1.1e-1 (f); 6.1e-1 -> | Helped |
| Add exploitation alongside exploration (ablation) | SGA with bilevel optimization · Constitutive law (d): weakly compressible fluid | loss: 1.4e-12 -> 1.4e-12 | No clear effect |
| Add exploitation alongside exploration (ablation) | SGA with bilevel optimization · Molecule design (h): HOMO-LUMO gap target -2 | loss: 2.8e-5 -> 3.6e-5 | Hurt |
| Increase iterations from 5 to 20 (ablation) | SGA · Constitutive law (a–d) and molecule design (e–h) | loss: 5.2e-5 -> 4.2e-6 (a); 2.1e-1 -> 4.0e-4 (b); 6.0e-2 -> 2.5e-3 (c); 1.4e-12 -> 1.4e-12 (d); 1.3e-4 -> 1.3e-4 (e); 1.1e-1 - | Mixed |
| Use GPT-4 instead of the other tested LLM backbones (ablation) | SGA · Constitutive law search and molecule design | rank | Helped |
| Use a different backbone, Mixtral-8x7B instead of GPT-4 (ablation) | SGA · Two molecule design tasks | rank | Mixed |
Surprising findings
- Generated constitutive laws and molecules were judged by domain experts as unconventional yet reasonable, and an imaginary constitutive law mixture was still fit with low loss, suggesting not mere memorization.
- Mixtral-8x7B topped two molecule design tasks, beating GPT-4 on those tasks.
Limitations
- LLM-generated code is executed without filtering, posing safety risks.
- Performance depends on differentiability of generated code; interpretability of LLM solutions is hard to ensure.
- High LLM inference cost (around $10 per task with GPT-4).
Experimental setup
Datasets: QM9 SRBench
Benchmarks: MPM constitutive law tasks (a-d) QM9 molecule design tasks (e-h) SRBench
Metrics: loss (MSE) R2 MSE MAE
Baselines: Chain-of-Thought FunSearch Eureka OPRO GhemGE SRBench methods SymbolicGPT NeSymReS T-JSL
Builds on: OPRO FunSearch Eureka warp RDKit
Closest papers by meaning
- PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors (NeurIPS 2025 · similarity 0.90)
- LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models (ICML 2025 · similarity 0.87)
- Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agents (ICML 2026 · similarity 0.86)
- MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem (NeurIPS 2025 · similarity 0.86)
- ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows (ICLR 2026 · similarity 0.86)
- MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.