Research map / Learning theory and algorithms / Combinatorial and search optimization
End-to-end Symbolic Regression with Transformers
Pierre-Alexandre Kamienny, Stéphane d'Ascoli, Guillaume Lample, Francois Charton
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| predict full expression with constants (E2E) instead of skeleton only, both with BFGS refinement (ablation) | Transformer symbolic regression · In-domain validation | Acc_0.001: 0.17 -> 0.29 | Improved |
| add BFGS refinement initialized from model-predicted constants (ablation) | E2E model · In-domain validation | Acc_0.001: 0.09 -> 0.29 | Improved |
| E2E without refinement vs skeleton + BFGS (ablation) | E2E model · In-domain validation | R^2 / Acc_0.001: R^2 0.43 -> 0.62; Acc_0.001 0.17 -> 0.09 | Mixed |
| initialize BFGS with model predictions instead of random init (ablation) | E2E + BFGS · In-domain validation | R^2: 0.44 -> 0.68 | Improved |
| BFGS refinement with random init instead of no refinement (ablation) | E2E model · In-domain validation | R^2: 0.62 -> 0.44 | Hurt |
| increase number of input points N (ablation) | E2E and skeleton models · In-domain validation | accuracy | Improved |
| increase target noise without refinement (ablation) | E2E no BFGS · In-domain validation | accuracy | Hurt |
| use beam search instead of sampling for decoding candidates (ablation) | E2E model without refinement · SRBench black-box | median R^2 | Hurt |
Closest papers by meaning
- Transformer-based model for symbolic regression via joint supervised learning (ICLR 2023 · similarity 0.91)
- Contemporary Symbolic Regression Methods and their Relative Performance (NeurIPS 2021 · similarity 0.91)
- ParFam -- (Neural Guided) Symbolic Regression via Continuous Global Optimization (ICLR 2025 · similarity 0.88)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.88)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.88)
- MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
