Research map / Large language models / Agents, code and math
Symbolic Regression with a Learned Concept Library
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaSR, which augments PySR-style genetic symbolic regression with LLM-guided operators conditioned on a self-evolving library of natural-language concepts, outperforming prior SR methods on Feynman equations.
Problem
Symbolic regression searches an exponentially large discrete space of programs; purely symbolic genetic search is slow and lacks prior knowledge about which structures are plausible.
Approach
Induce a library of abstract natural-language concepts by prompting an LLM to summarize high-performing and low-performing hypotheses, evolve those concepts, and use them zero-shot to guide LLM-based mutation, crossover and initialization interleaved with standard genetic operators.
Key results
- LaSR solves 72/100 Feynman equations exactly versus 59/100 for PySR and 40/100 for the best non-PySR baseline (uDSR).
- On the synthetic leakage-check dataset, LaSR (Llama3-8B, p=0.1%) reaches mean test R^2 0.913 versus 0.070 for PySR.
- LaSR discovers a three-parameter scaling law on a BIG-Bench subset with MSE 0.03598, matching Chinchilla (0.03598 with 5 parameters).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LaSR instead of PySR | Symbolic regression on Feynman Equations · Feynman Equations | exact match solve rate: 59/100 -> 72/100 | Helped |
| Increase Llama3-8B LLM-operation probability from p=1% to p=5% | LaSR · Feynman Equations | exact solve: 67/100 -> 69/100 | Helped |
| Increase Llama3-8B LLM-operation probability from p=5% to p=10% | LaSR · Feynman Equations | exact solve: 69/100 -> 71/100 | Helped |
| Use GPT-3.5 instead of Llama3-8B at p=1% | LaSR · Feynman Equations | exact solve: 67/100 -> 72/100 | Helped |
| Remove concept evolution (ablation) | LaSR · Feynman Equations | MSE solve rate | Hurt |
| Remove the concept library (ablation) | LaSR · Feynman Equations | MSE solve rate | Hurt |
| Remove variable names (ablation) | LaSR · Feynman Equations | MSE solve rate | Hurt |
| Add vague user-provided natural-language hints (ablation) | LaSR · Feynman Equations | MSE solve rate | Helped |
| Use LaSR instead of PySR | Symbolic regression on synthetic equations · Synthetic symbolic regression dataset | test set R^2: 0.070 -> 0.913 | Helped |
| Use the LaSR-discovered scaling-law skeleton instead of the Chinchilla skeleton | BigBench multiple choice grade prediction · BigBench multiple choice grade | MSE loss: $0.03610 \pm 0.00268$ -> $0.03598 \pm 0.00265$ | No clear effect |
| Add number of shots to the training-steps exponent (ablation) | Modified Chinchilla scaling law · BigBench multiple choice grade | MSE loss: $0.03610 \pm 0.00268$ -> $\mathbf{0.03598 \pm 0.00264}$ | No clear effect |
| Replace a scaling-law skeleton with a residual-only constant score (ablation) | BigBench multiple choice grade prediction · BigBench multiple choice grade | MSE loss: $0.03598 \pm 0.00265$ -> $0.09324 \pm 0.01992$ | Hurt |
Surprising findings
- Even minimal LLM guidance (Llama3-8B at p=1%) outperforms PySR on Feynman equations.
- PySR run for 10 hours per equation with up to 1e6 iterations solves only 3 more equations, still well below LaSR's 40-iteration results.
Limitations
- Cannot guarantee that learned concepts are correct, insightful, or mutually consistent.
- Performance is bottlenecked by the backbone LLM's reasoning ability and call frequency; higher-compute regimes not tested.
- Scaling-law result is preliminary and the discovered equation has an parity-related generalization issue.
Experimental setup
Datasets: Feynman Equations BIG-Bench
Benchmarks: Feynman Equations synthetic equations dataset BIG-Bench scaling law subset
Metrics: exact solve rate test set R^2 MSE loss
Baselines: GPlearn AFP AFP-FE DSR uDSR AI Feynman PySR
Builds on: PySR FunSearch AI Feynman
Closest papers by meaning
- Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search (ICML 2023 · similarity 0.89)
- A Unified Framework for Deep Symbolic Regression (NeurIPS 2022 · similarity 0.89)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.88)
- FunctionEvolve: Structure-Guided Symbolic Regression with LLMs (NeurIPS 2026 · similarity 0.88)
- Improving Monte Carlo Tree Search for Symbolic Regression (NeurIPS 2025 · similarity 0.87)
- LLM-SR: Scientific Equation Discovery via Programming with Large Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.