Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Pretrained Optimization Model for Zero-Shot Black Box Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Zero-shot black-box optimization: optimizers that work on unseen tasks and dimensions without per-task hyperparameter tuning.
Approach
Replace hand-designed evolutionary operators with a learned population-based optimizer: a multi-head self-attention module generates mutation weights from fitness and rank, and a feed-forward module with Gumbel-softmax produces per-individual crossover probabilities; the whole loop is unrolled and trained end-to-end by gradient descent on simple parametric functions (MetaGBT).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply POM directly instead of the compared optimizers | BBOB · BBOB, d=30 and d=100 | mean ranking score | Helped |
| apply POM instead of LSHADE | BBOB F1 · BBOB F1, d=100 | objective value: 1.64E+01(8.78E-01) -> 5.92E-12(5.92E-12) | Helped |
| apply POM instead of LSHADE | BBOB F1 · BBOB F1, d=500 | objective value: 1.28E+02(1.87E+01) -> 1.98E-12(1.98E-12) | Helped |
| add LMM instead of using DE/rand/1/bin mutation (ablation) | POM · BBOB d=30 | mean ranking score: 2.1667 -> 4.3333 | Helped |
| add learnable crossover (LCM) instead of binomial crossover (ablation) | POM · BBOB d=30 | mean ranking score: 4.0000 -> 4.3333 | Helped |
| train POM instead of using random initialization (ablation) | POM · BBOB d=30 | mean ranking score: 2.9167 -> 4.3333 | Helped |
| add the mask operation (ablation) | POM · BBOB d=30 | mean ranking score: 1.5833 -> 4.3333 | Helped |
| increase model size to XL (ablation) | POM · BBOB d=100 | mean ranking score | Helped |
| increase the number of basic functions in the training set (ablation) | POM · BBOB d=100 | mean ranking score | Helped |
| introduce composite functions into the training set (ablation) | POM · BBOB d=100 | mean ranking score | Hurt |
| fine-tune POM using a small number of samples (ablation) | POM · TF6–TF8 | relative performance improvement | Helped |
| use a larger POM architecture (ablation) | POM · BBOB d=100 | mean ranking score | Mixed |
Experimental setup
Datasets: BBOB Bipedal Walker Enduro synthetic training functions TF1-TF5
Benchmarks: BBOB (d=30, 100, 500) Bipedal Walker Enduro
Metrics: best objective value found win/tie/loss critical difference diagram relative performance improvement total reward
Baselines: DE ES L-SHADE CMA-ES LES LGA TurBO
Builds on: Differential Evolution LES LGA Gumbel-softmax Multi-head self-attention
Closest papers by meaning
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.84)
- In-Context Black-Box Optimization with Unreliable Feedback (NeurIPS 2026 · similarity 0.83)
- SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning (ICLR 2024 · similarity 0.83)
- A Generalization Theory for Zero-Shot Prediction (ICML 2025 · similarity 0.83)
- MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization (NeurIPS 2025 · similarity 0.83)
- Zero-Shot Transfer of Neural ODEs (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.