Research map / Learning theory and algorithms / Combinatorial and search optimization
Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SR4MDL, an MCTS-based symbolic regression method guided by MDLformer, a Transformer that predicts a formula's minimum description length (formula length) from data, replacing prediction error as the search objective.
Problem
Heuristic symbolic regression searches optimize prediction error, which does not decrease monotonically toward the target formula, leading to low recovery rates of the true formula.
Approach
Replace prediction-error search objective with a learned estimate of minimum description length (formula complexity) produced by a cross-modally aligned Transformer (MDLformer), so that MDL decreases monotonically along the path to the target formula; plug this into MCTS with a PUCT rule.
Key results
- Recovers around 50 of 133 ground-truth formulas (Strogatz 66.78% and Feynman 33.93% recovery rate averaged over seeds and noise levels), outperforming the best baseline by 43.92%.
- MDLformer predicts formula length with RMSE 3.91, R^2 0.9035, AUC 0.8859 on 1024 synthetic formula-data pairs.
- On black-box PMLB problems, achieves Pareto rank 1 with Operon, GP-GOMEA, and DSR (test R^2 0.6258 at complexity 29.88).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MDLformer-guided search instead of the baseline search objectives | Symbolic regression on Strogatz · Strogatz | Recovery rate: 15.27% -> 66.78% | Helped |
| Use MDLformer-guided search instead of the baseline search objectives | Symbolic regression on Feynman · Feynman | Recovery rate: 27.24% -> 33.93% | Helped |
| Use MDL-guided search rather than baseline methods | Symbolic regression on PMLB black-box problems · PMLB black-box test set | Test R²: 0.7945 -> 0.6258 | Mixed |
| Add alignment training before prediction training (ablation) | MDLformer · MDLformer training-strategy ablation | RMSE | Helped |
| Train alignment and prediction objectives concurrently instead of sequentially (ablation) | MDLformer · MDLformer training-strategy ablation | RMSE | Hurt |
| Remove alignment training and train only with the prediction objective (ablation) | MDLformer · MDLformer training-strategy ablation | RMSE | Hurt |
| Increase the number of input data points (ablation) | MDLformer · MDLformer input-point ablation | Prediction error | Helped |
| Increase feature-noise intensity (ablation) | MDLformer · MDLformer feature-noise ablation | AUC | No clear effect |
| Increase feature-noise intensity (ablation) | MDLformer · MDLformer feature-noise ablation | RMSE | Hurt |
Surprising findings
- AIFeynman2 has higher recovery than the proposed method on noise-free Feynman problems, attributed to its hand-designed rules for that dataset.
- Generative and regression neural methods have much lower recovery rates than search-based methods despite being faster.
Limitations
- MDLformer performance degrades on data with complex relationships.
- Slower than generative methods at inference.
- Black-box results show lower test R^2 than some baselines (e.g., Operon) on PMLB.
Experimental setup
Datasets: SRbench Strogatz (14 problems) SRbench Feynman (119 problems) PMLB black-box (122 problems)
Benchmarks: SRbench Strogatz SRbench Feynman PMLB black-box
Metrics: recovery rate search time test R^2 formula complexity RMSE R^2 AUC
Baselines: FEAT NeurSR E2ESR SNIP GPlearn AFP AFP-FE EPLEX SBP-GP GP-GOMEA Operon SPL
Builds on: Monte Carlo tree search PUCT SPL AI Feynman SNIP
Closest papers by meaning
- Symbolic Regression with a Learned Concept Library (NeurIPS 2024 · similarity 0.88)
- ParFam -- (Neural Guided) Symbolic Regression via Continuous Global Optimization (ICLR 2025 · similarity 0.88)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.87)
- FunctionEvolve: Structure-Guided Symbolic Regression with LLMs (NeurIPS 2026 · similarity 0.86)
- Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large $p$ (ICML 2025 · similarity 0.86)
- A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.