Research map / Learning theory and algorithms / Combinatorial and search optimization
MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaSymNet, a tree-like symbolic regression network whose PANGU meta-activation functions and variable meta-functions evolve into basic symbols during gradient-based training, with gradient-driven structural growth/reduction.
Problem
Symbolic regression is usually treated as combinatorial search (GP, RL), which degrades on complex tasks; EQL-style neural approaches use fixed structures and struggle with sparsification.
Approach
Reformulate symbolic regression as numerical optimization over a tree-structured network whose nodes use a softmax-weighted mixture of candidate symbols (PANGU meta-function) that is gradually collapsed to one symbol, with variable meta-functions for leaves and an entropy term in the loss that encourages one-hot selection; the tree grows or shrinks as activations change.
Key results
- Average R^2 of 0.9859 across benchmark groups, versus 0.9749 for DSO and 0.7852 for EQL (Table 1).
- Average expression complexity of 25.3 nodes, second lowest after TPSR at 24.4 (Table 2).
- Entropy loss ablation: average R^2 0.9859 with entropy loss vs 0.9743 without.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add entropy loss (ablation) | MetaSymNet · All datasets (average) | R^2: 0.9743 -> 0.9859 | Helped |
| add entropy loss (ablation) | MetaSymNet · All datasets (average) | Nodes: 29.7 -> 25.3 | Helped |
| add entropy loss (ablation) | MetaSymNet · All datasets (average) | Time (s): 133 -> 115 | Helped |
Surprising findings
- MetaSymNet is slightly lower than DSO in full recovery rate overall, though better on some complex datasets.
- On extrapolation to [-5,5] for Nguyen-12, MetaSymNet's R^2 is negative (-1.818), and the average extrapolation R^2 is 0.6888.
Limitations
- Tuning the entropy loss coefficient lambda is difficult.
- Can get trapped in local optima, yielding approximate rather than exact expressions.
- Greedy selection strategy for function evolution.
Experimental setup
Datasets: Nguyen Keijzer Korns Constant Livermore R Vladislavleva Jin Neat Others SRBench (Feynman, Strogatz, Black-box) SRSD-Feynman
Benchmarks: Nguyen Keijzer Korns Constant Livermore R Vladislavleva Jin Neat Others SRBench Feynman SRBench Strogatz
Metrics: R^2 inference time expression complexity (nodes) NED full recovery rate noise robustness
Baselines: DSO TPSR SPL NeSymReS EQL MLP SVR
Builds on: Equation Learner (EQL) Deep Symbolic Regression DSO AI Feynman
Closest papers by meaning
- SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning (ICLR 2024 · similarity 0.88)
- A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data (ICML 2024 · similarity 0.88)
- Neuro-Guided Graph Search for Symbolic Regression (Student Abstract) (AAAI 2025 · similarity 0.87)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.85)
- Symbolic Regression with a Learned Concept Library (NeurIPS 2024 · similarity 0.85)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.