Research map / Probabilistic and scientific modeling / Bayesian optimization
SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SYMBOL, a MetaBBO framework where an LSTM-based Symbolic Equation Generator produces closed-form black-box optimizer update rules, meta-trained with three RL strategies (exploration, teacher-guided, synergized).
Problem
Existing meta-learned black-box optimizers either only tune hand-crafted optimizers (limiting flexibility) or generate candidate solutions with opaque neural networks (poor interpretability, scaling and generalization issues).
Approach
Generate interpretable closed-form update rules autoregressively as pre-order traversals of symbolic expression trees over a basis symbol set of first-principle BBO operands, with on-the-fly constant inference, conditioned on fitness landscape analysis features and trained with PPO under three reward designs (exploration, guided by a teacher optimizer via EMD-style reward, and a synergy of both).
Key results
- SYMBOL-S reaches 0.972 mean normalized performance on the meta-test distribution, the best among all baselines (MadDE 0.940, LDE 0.970, ipop-CMA-ES 0.970).
- On HPO-B meta-generalization, SYMBOL-S scores 0.963 versus SMAC 0.979 and Meta-ES 0.949, while solving in under 1 s per instance versus SMAC's ~42 s.
- On protein-docking, SYMBOL-S scores 0.999, the best overall (SMAC 0.998), with about 0.7 s per instance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine exploration and teacher-guided objectives in Symbol-S instead of using Symbol-E | Symbol · CEC-competition (meta test) | mean normalized performance: 0.934 -> 0.972 | Helped |
| combine exploration and teacher-guided objectives in Symbol-S instead of using Symbol-G | Symbol · CEC-competition (meta test) | mean normalized performance: 0.964 -> 0.972 | Helped |
| remove population features from FLA (ablation) | Symbol-S · CEC-competition | mean normalized performance: 0.926 vs 0.972 | Hurt |
| remove objective features from FLA (ablation) | Symbol-S · HPO-B | mean normalized performance: 0.875 vs 0.963 | Hurt |
| remove timestamp features from FLA (ablation) | Symbol-S · HPO-B | mean normalized performance: 0.779 vs 0.963 | Hurt |
| replace FLA features with coordinate features (ablation) | Symbol-S · CEC-competition | mean normalized performance: 0.849 vs 0.972 | Hurt |
| replace FLA features with coordinate features (ablation) | Symbol-S · HPO-B | mean normalized performance | Hurt |
| replace FLA features with coordinate features (ablation) | Symbol-S · Protein-docking | mean normalized performance | Hurt |
| augment the basis symbol set with S+ (ablation) | Symbol-S · CEC-competition | mean normalized performance: 0.972 -> 0.952 | Hurt |
| augment the basis symbol set with S+ (ablation) | Symbol-S · HPO-B | mean normalized performance: 0.963 -> 0.937 | Hurt |
| augment the basis symbol set with S+ (ablation) | Symbol-S · Protein-docking | mean normalized performance: 0.999 -> 0.995 | Hurt |
| remove symbols to form S- (ablation) | Symbol-S · CEC-competition | mean normalized performance: 0.972 -> 0.930 | Hurt |
Surprising findings
- MetaBBO baselines RNN-Opt and MELBA fail to generalize across dimensions and LDE across population sizes, and DEDDQN and Meta-ES can perform worse than random search on realistic tasks.
- SYMBOL-G fails to learn from PSO as a teacher, which the authors attribute to overly definitive update rules causing overfitting.
Limitations
- Selecting a suitable teacher optimizer for new BBO tasks can be challenging, and SYMBOL-E is less sample-efficient without a teacher.
- The basis symbol set is fixed and hand-designed; more powerful symbol sets are left for future work.
- SYMBOL-G and SYMBOL-S do not consistently learn from every teacher (e.g., PSO), limiting teacher-agnosticism.
Experimental setup
Datasets: IEEE CEC numerical optimization benchmark (synthesized training distribution) HPO-B Protein-docking benchmark
Benchmarks: CEC synthetic meta-test distribution HPO-B Protein-docking
Metrics: min-max normalized performance (mean) runtime rank
Baselines: Random Search MadDE sep-CMA-ES ipop-CMA-ES SMAC DEDDQN LDE Meta-ES MELBA RNN-Opt
Builds on: MetaBBO PPO MadDE Symbolic-L2O SMAC
Closest papers by meaning
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.89)
- MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization (NeurIPS 2025 · similarity 0.89)
- MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions (AAAI 2025 · similarity 0.88)
- DisCo-DSO: Coupling Discrete and Continuous Optimization for Efficient Generative Design in Hybrid Spaces (AAAI 2025 · similarity 0.87)
- Improving Monte Carlo Tree Search for Symbolic Regression (NeurIPS 2025 · similarity 0.86)
- Towards General Algorithm Discovery for Combinatorial Optimization: Learning Symbolic Branching Policy from Bipartite Graph (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.