Research map / Probabilistic and scientific modeling / Molecules and proteins
ParFam -- (Neural Guided) Symbolic Regression via Continuous Global Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ParFam, a continuous-optimization symbolic regression method using rational-layer parametric families solved with basin-hopping, plus DL-ParFam which uses a pre-trained Set Transformer to predict family structure.
Problem
Symbolic regression requires searching a discrete space of formulas; existing continuous-optimization SR methods (e.g., EQL) struggle with local minima and restricted expressivity, and many SR methods are slow or unreliable on SRBench ground-truth problems.
Approach
Translate SR into continuous optimization via a single-hidden-layer residual-like parametric family in which linear connections are replaced by rational functions and unary physical base functions (sin, exp, sqrt) are used; optimize coefficients with basin-hopping plus L-BFGS fine-tuning, and predict the family structure with a pre-trained Set Transformer (DL-ParFam).
Key results
- ParFam is claimed to achieve the highest accuracy solution rate with and without noise on SRBench ground-truth problems, with PySR best on symbolic solution rate among the top methods followed by ParFam.
- DL-ParFam accelerates the optimization by up to two orders of magnitude (up to 100x) relative to ParFam-style search, while performing slightly worse than ParFam in accuracy.
- Theory gives c_l/d_l ≈ (r2/x1)^l, e.g. about 90.25% expressivity coverage at complexity 5 for n=4, k=3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ParFam instead of the other evaluated methods | SRBench ground-truth problems · SRBench ground-truth problems, with and without noise | accuracy solution rate | Helped |
| Use ParFam instead of competing methods | SRBench ground-truth problems · SRBench ground-truth problems | symbolic solution rate | Mixed |
| Add pretrained neural guidance to predict ParFam model parameters | ParFam · SRBench ground-truth problems | training time: up to a hundred times faster | Helped |
| Add pretrained neural guidance to predict ParFam model parameters | ParFam · SRBench ground-truth problems | symbolic solution rate | Hurt |
| Add pretrained neural guidance to predict ParFam model parameters | ParFam · SRBench ground-truth problems | accuracy solution rate | Hurt |
| Use ParFam on black-box problems | ParFam · SRBench black-box problems (77 problems, at most 10 variables) | median R² | Mixed |
| Use basin-hopping rather than the other tested optimizers (ablation) | ParFam · Feynman subset (15 problems) | symbolic solution rate | Helped |
| Replace basin-hopping with BFGS multi-start (ablation) | ParFam · Feynman subset (15 problems) | symbolic solution rate | No clear effect |
| Replace basin-hopping with BFGS multi-start (ablation) | ParFam · Feynman subset (15 problems) | accuracy solution rate | No clear effect |
| Use the pretrained Set Transformer on Feynman rather than synthetic data | DL-ParFam parameter predictor · Feynman and synthetic datasets | top-3 correct model-parameter rate: 50.2% -> 38.0% | Hurt |
Surprising findings
- DL-ParFam's symbolic solution rate is notably hindered under low-noise conditions.
- Top-1 accuracy of the Set Transformer on Feynman problems (30.4%) is only moderately below its synthetic accuracy (31.4%), but the paper notes room to improve synthetic training data.
Limitations
- Parametric structure limits the function space; high-dimensional problems (>10 variables) are costly for global optimizers.
- DL-ParFam requires costly pre-training and is less flexible than ParFam; limited to up to 9 variables.
Experimental setup
Datasets: SRBench Feynman SRBench Strogatz SRBench black-box datasets synthetic functions for DL-ParFam pre-training
Benchmarks: SRBench Feynman SRBench Strogatz SRBench black-box
Metrics: symbolic solution rate accuracy solution rate (R^2>0.999) median R^2 formula complexity training time
Baselines: PySR AI Feynman uDSR EndToEnd Operon MRGP EQL NeSymRes SNR
Builds on: EQL SRBench basin-hopping SNR NeSymRes
Closest papers by meaning
- Contemporary Symbolic Regression Methods and their Relative Performance (NeurIPS 2021 · similarity 0.90)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.88)
- Pareto-Optimal Fronts for Benchmarking Symbolic Regression Algorithms (ICML 2025 · similarity 0.86)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.86)
- Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large $p$ (ICML 2025 · similarity 0.85)
- Symbolic Regression with a Learned Concept Library (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.