Research map / Probabilistic and scientific modeling / Molecules and proteins
Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large $p$
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PAN+SR, a BART-based nonparametric variable pre-screening step (VIP rank clustering with AHC) that reduces high-dimensional inputs before symbolic regression, and extends SRBench with high-dimensional black-box and noisy Feynman problems.
Problem
Symbolic regression does not scale to datasets with many input variables (large p), many of which are irrelevant, and with measurement noise; the search space grows double-exponentially and variable selection must have near-zero false negative rate.
Approach
Average BART variable-inclusion-proportion rankings over K independent runs, then cluster the average ranks with agglomerative hierarchical clustering into relevant and irrelevant groups, selecting features without tuning or knowledge of sparsity; the pre-screened data is then fed to any SR method (parametric-assisted nonparametric strategy).
Key results
- PAN+SR improved test R^2 for 18 of 19 SR methods on black-box datasets; PAN reduced AIFeynman's training time from 71250 s to 13997 s.
- On ground-truth Feynman problems (n=1000, noiseless), PAN+SR improved symbolic solution rate for all 19 methods; uDSR solution rate rose from 36.6% to 71.8%.
- PAN achieved near 0% FNR in most settings, with FNR above 5% only at n=500 and SNR=0.5; average PAN runtime was 74.14 s on black-box and 325 s on ground-truth problems.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PAN pre-screening before SR | 19 symbolic regression methods · Black-box regression datasets | test R² | Helped |
| Apply PAN pre-screening before SR | 19 symbolic regression methods · Feynman ground-truth datasets, noiseless | solution rate | Helped |
| Apply PAN pre-screening before uDSR | uDSR · Feynman ground-truth, n=1000, SNR=∞, s=50 | solution rate: 36.6% -> 71.8% | Helped |
| Apply PAN pre-screening before uDSR | uDSR · Feynman ground-truth, n=1000, SNR=∞, s=50 | training time: 9573 seconds to 1596 seconds | Helped |
| Apply PAN pre-screening before AIFeynman | AIFeynman · Black-box regression datasets | training time: 71250 seconds to 13997 seconds | Helped |
| Apply PAN pre-screening before DSR | DSR · Feynman ground-truth, n=1000, SNR=10 | solution rate: 8.2% to 14.9% | Helped |
| Apply PAN pre-screening before DSR | DSR · Feynman ground-truth, n=1000, SNR=∞ | solution rate: 8.9% to 25.8% | Helped |
| Use VIP Rank/PAN instead of BART-G.SE for pre-screening before Operon (ablation) | Operon · Feynman datasets, n=500, SNR=20 | R²: 0.860 -> 0.892 | Helped |
| Use VIP Rank/PAN instead of BART-G.SE for pre-screening before Operon (ablation) | Operon · Feynman datasets, n=500, SNR=0.5 | R²: 0.142 -> 0.145 | Helped |
| Use agglomerative hierarchical clustering rather than other clustering algorithms for VIP Rank (ablation) | VIP Rank · Feynman datasets across sample sizes and SNR levels | FPR | Helped |
| Use alternative clustering algorithms beyond the top five for VIP Rank (ablation) | VIP Rank · Feynman datasets across simulation settings | TPR | Hurt |
| Add independent Gaussian noise to predictors (ablation) | VIP Rank · Friedman equation, n=1000, p=100, SNR=10 | FPR: 10.58% -> 26.42% | Hurt |
Surprising findings
- Standalone AIFeynman ran out of memory and failed to complete any ground-truth run, but PAN lifted it to 2nd overall in symbolic solution rate.
- PAN's false positive rate rose from 0% at noiseless to over 30% at SNR=0.5, a tradeoff accepted because false negatives are prioritized.
Limitations
- PAN's false negative rate rises above 5% at n=500 and SNR=0.5, and false positive rate grows substantially under heavy noise.
- All SR methods, including PAN-boosted ones, degrade sharply at SNR=10; the paper does not solve low-SNR SR.
- Experiments use cluster CPU hardware with 16 GB limits and synthetic Feynman data with uniform iid features and additive Gaussian noise.
Experimental setup
Datasets: PMLB high-dimensional regression datasets (35) Extended Feynman Symbolic Regression Database (100 equations) SRBench
Benchmarks: PMLB high-dimensional black-box regression (35 datasets) Extended Feynman ground-truth problems (100 equations)
Metrics: test R^2 symbolic solution rate model complexity training time true positive rate false positive rate false negative rate
Baselines: Operon PySR uDSR AIFeynman 2.0 BART-G.SE BART-Local BART-G.MAX Random forest variable selection gplearn AFP AFP_FE EHC
Builds on: SRBench PySR BART-G.SE PAN
Closest papers by meaning
- Contemporary Symbolic Regression Methods and their Relative Performance (NeurIPS 2021 · similarity 0.89)
- Pareto-Optimal Fronts for Benchmarking Symbolic Regression Algorithms (ICML 2025 · similarity 0.88)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.87)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.86)
- FunctionEvolve: Structure-Guided Symbolic Regression with LLMs (NeurIPS 2026 · similarity 0.86)
- Dimension Reduction for Symbolic Regression (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.