Research map / Learning theory and algorithms / Prediction and decision losses

B2Opt: Learning to Optimize Black-box Optimization with Little Budget

AAAI 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Determinantal point process inference · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes B2Opt, a Transformer-style learned genetic-algorithm optimizer (self-attention crossover, FFN mutation, residual selection) trained with gradients on cheap differentiable surrogate functions, so no expensive black-box queries are needed during training.

Problem

High-dimensional, expensive black-box optimization needs strategies that reach good solutions with very few function evaluations; prior learned optimizers either have weak strategy representations or require many expensive evaluations to train.

Approach

Parameterize GA operators (crossover, mutation, selection) as learnable Transformer-like modules (SAC, FM, RSSM) stacked into blocks, and train them by gradient descent on cheap differentiable surrogate functions (F1-F3) with shifted biases, avoiding querying the target function during training.

Key results

• On the planar mechanical arm complex case at r<=100, B2Opt reaches 0.17(0.01) versus 0.25 for L-SHADE and 0.72 for IPOP-CMA-ES, winning all six settings against the baselines except one L-SHADE loss in the simple case at r=1000.
• On MNIST with full data, B2Opt reaches 0.86 test accuracy versus 0.67 for CMA-ES and 0.30 for L-SHADE, using 144 evaluations versus 3000 for L-SHADE.
• B2Opt with 30 OBs and weight sharing has the best results on F4-F9 at d=100, while at d=10 it loses to L-SHADE in five of the cases.

Measured effects reported in the paper

Surprising findings

• The untrained B2Opt outperforms most baselines on the planar mechanical arm task, suggesting the architecture alone provides useful search bias.
• Deeper B2Opt (5 OBs without weight sharing) performs better than shallower weight-shared variants (3 OBs), but deeper models are harder to train.

Limitations

• Performance degrades sharply when the surrogate training set has low similarity to the target optimization objective.
• Training deep B2Opt architectures without weight sharing is difficult; learning rate is sensitive and hyperparameters were only coarsely tuned.
• Loss function does not explicitly account for population diversity.

Experimental setup

Datasets: F1-F3 synthetic surrogate functions MNIST (1000 training and 1000 test samples) planar mechanical arm target points (600 training, 128 test)

Benchmarks: F4-F9 synthetic functions (Sphere, Rosenbrock, Rastrigin, Griewank, Ackley, etc.) at d=10 and d=100 planar mechanical arm (simple and complex cases) MNIST small CNN neuroevolution

Metrics: objective value (lower is better) win/tie/loss counts classification test accuracy

Baselines: DE ES CMA-ES IPOP-CMA-ES L-SHADE Dragonfly SAASBO L2O-Swarm LES LGA

Builds on: Transformer genetic algorithm LES LGA L2O-Swarm

Closest papers by meaning

• AutoOpt: A Dataset and a Unified Framework for Automating Optimization Problem Solving (NeurIPS 2025 · similarity 0.85)
• SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning (ICLR 2024 · similarity 0.84)
• BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems (NeurIPS 2026 · similarity 0.84)
• Fast T2T: Optimization Consistency Speeds Up Diffusion-Based Training-to-Testing Solving for Combinatorial Optimization (NeurIPS 2024 · similarity 0.84)
• BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning (NeurIPS 2024 · similarity 0.84)
• Learning Surrogates for Offline Black-Box Optimization via Gradient Matching (ICML 2024 · similarity 0.83)