Research map / Learning theory and algorithms / Kernels and regression theory
Sample-efficient Bayesian Optimisation Using Known Invariances
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends Bayesian optimisation to objectives invariant to a known finite group by building totally invariant kernels, proving sample-complexity upper and lower bounds that scale with 1/|G|, and applying it to a nuclear fusion current-drive design task.
Problem
Bayesian optimisation of costly functions that are invariant to a known group of transformations (e.g. permutations of actuator launchers), where standard BO and constrained BO are sample-inefficient.
Approach
Construct the RKHS of G-invariant functions via a symmetrised kernel k_G(x,y)=1/|G| sum_sigma k(sigma(x),y) and use it inside GP-based BO (UCB, MVR), giving invariance-aware algorithms with theoretical sample-complexity bounds that improve by a factor of 1/|G|; also study partial (subgroup) invariance and quasi-invariance via additive kernels.
Key results
- Invariant-kernel UCB/MVR significantly outperforms non-invariant baselines on synthetic invariant tasks; on PermInv-6D baselines fail to find the optimum while permutation-invariant versions find it.
- Partial invariance (2- and 3-block subgroups) achieves most of the sample-efficiency gain of the full invariant kernel at about half the fitting cost.
- On the fusion task, the invariant UCB achieves significantly better performance than non-invariant UCB, which fails to find the high-performance region.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a permutation-invariant kernel instead of a standard kernel | Bayesian optimisation on PermInv-2D · PermInv-2D | sample efficiency | Helped |
| use a permutation-invariant kernel instead of a standard kernel | Bayesian optimisation on PermInv-6D · PermInv-6D | sample efficiency | Helped |
| use a cyclic-invariant kernel instead of a standard kernel | Bayesian optimisation on CyclInv-3D · CyclInv-3D | sample efficiency | Helped |
| constrain acquisition optimisation to the permutation group's fundamental region rather than using an invariant kernel | Constrained Bayesian optimisation · PermInv tasks | sample efficiency | Hurt |
| use a subgroup-invariant kernel rather than a standard kernel (ablation) | Bayesian optimisation on PermInv-6D · PermInv-6D | sample efficiency | Helped |
| use the fully invariant kernel rather than a non-invariant kernel | Bayesian optimisation on quasi-invariant functions · Synthetic quasi-invariant functions | performance | Helped |
| use a partially specified 3-block invariant kernel rather than a non-invariant kernel | UCB for nuclear fusion scenario design · Nuclear fusion scenario design | optimisation performance | Helped |
| use a partially specified 3-block invariant kernel rather than a fully invariant kernel (ablation) | Gaussian process hyperparameter fitting on PermInv-6D · PermInv-6D | computation time: by a factor of 2 | Helped |
| use a 2-block invariant kernel rather than a fully invariant kernel (ablation) | Gaussian process hyperparameter fitting on PermInv-6D · PermInv-6D | computation time: by a factor of 2 | Helped |
Surprising findings
- Constrained BO (restricting the search to a fundamental domain) is identical to standard BO in high dimensions, despite the invariance.
- Upper and lower sample-complexity bounds differ in the exponent of 1/|G| (1+(d-1)/nu vs 1+(d-1)/(2nu)), leaving a gap.
Limitations
- Computational cost of the group sum becomes a bottleneck for very large groups (e.g. 12! for the fusion task), motivating subgroup approximations.
- Theoretical bounds rely on an assumption on eigenvalue multiplicities that excludes some groups in low dimensions (d=2,3).
- Gap between the upper and lower bound exponents of 1/|G|.
Experimental setup
Datasets: synthetic group-invariant Matern-5/2 functions (PermInv-2D, PermInv-6D, CyclInv-3D) nuclear fusion safety factor current-drive objective (STEP SPR45 design)
Benchmarks: PermInv-2D PermInv-6D CyclInv-3D nuclear fusion current drive task
Metrics: simple regret cumulative regret sample efficiency compute time and memory
Baselines: UCB MVR constrained Bayesian optimisation (CBO) non-invariant kernel BO
Builds on: Bayesian optimisation with GP-UCB maximum variance reduction constrained Bayesian optimisation invariant kernel ridge regression
Closest papers by meaning
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.88)
- A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.87)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.87)
- Domain Invariant Learning for Gaussian Processes and Bayesian Exploration (AAAI 2024 · similarity 0.87)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.87)
- Vanilla Bayesian Optimization Performs Great in High Dimensions (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.