Research map / Large language models / Agents, code and math

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs

NeurIPS 2026 · Large language models › Agents, code and math · approach: LLMs for scientific discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FunctionEvolve, an LLM-driven evolutionary symbolic regression framework with explicit AST-visible selection, mutation, and structure-aware coefficient fitting, reaching 107/129 exact forms on LLM-SRBench synthetic tasks with Claude Opus 4.6.

Problem

LLM-driven symbolic regression methods treat candidate expressions as opaque strings or programs, leading to poor exact symbolic recovery and unreliable coefficient fitting that discards correct skeletons.

Approach

Makes expression ASTs explicit across the whole search: structural summaries drive diverse parent selection, rule-based and LLM-guided local tree edits preserve useful subexpressions, and a structure-aware optimizer (variable projection, DE/CMA-ES, L-BFGS, snapping, simplification) fits coefficients per skeleton.

Key results

• With Claude Opus 4.6, FunctionEvolve recovers 107/129 exact forms at SA@50 (82.9%) and 72 at SA@1 (55.8%), versus 24 SA@50 for same-backbone LLM-SR and OpenEvolve.
• At top-1, FunctionEvolve reaches 72 SA@1, about 3.6x the 20 of published PiT-PO.
• On AI-Feynman's 120-task full set, FunctionEvolve recovers all 120 at top-1 (SA@1).

Surprising findings

• A No-LLM variant with GP-style rule mutation and structure-aware fitting already reaches 29 SA@50 and 16 SA@1, close to PiT-PO's top-1 result.
• The LLM-SRBench MatSci subset has near-perfect collinearity between temperature and strain (T ≈ 273 + 500ε), making some mechanisms unidentifiable.

Limitations

• Llama-3.1-8B shows weaker JSON-format following, so invalid outputs fall back to the rule-based baseline.
• Noisy observations sharply degrade exact symbolic recovery (107 to 54 at 1% noise).
• OOD paired comparison to ground truth is weaker (38.8% no worse than GT), and Acc_tau metrics are brittle for small y values.

Experimental setup

Datasets: LLM-SRBench synthetic subset (129 tasks) AI-Feynman (100 original + 20 bonus equations)

Benchmarks: LLM-SRBench synthetic subset (129 tasks) AI-Feynman

Metrics: SA@50 SA@1 Acc_0.1 NMSE Acc_0.01 Acc_0.001

Baselines: LLM-SR OpenEvolve LaSR PiT-PO PySR SR-Scientist

Builds on: LLM-SR OpenEvolve PySR LLM-SRBench

Closest papers by meaning

• LLM-SR: Scientific Equation Discovery via Programming with Large Language Models (ICLR 2025 · similarity 0.90)
• SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.89)
• Symbolic Regression with a Learned Concept Library (NeurIPS 2024 · similarity 0.88)
• Improving Monte Carlo Tree Search for Symbolic Regression (NeurIPS 2025 · similarity 0.87)
• MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.87)
• Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.86)