Research map / Probabilistic and scientific modeling / Bayesian optimization
Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SVH-PSL, which combines Stein Variational Gradient Descent with a hypernetwork-based Pareto set model and a per-objective local kernel to improve Pareto set learning for expensive multi-objective optimization under GP surrogates.
Problem
Pareto set learning for expensive multi-objective optimization relies on Gaussian process surrogates, whose fragmented uncertainty creates pseudo-local optima that trap LCB-driven search and cause premature convergence.
Approach
Sample multiple preference-conditioned solutions from a hypernetwork and update them jointly with SVGD, using a repulsive kernel term to keep particles diverse and escape pseudo-local optima, plus a local kernel that computes similarity per objective dimension with the maximizing (Chebyshev-active) objective.
Key results
- SVH-PSL converges faster than PSL-MOBO on synthetic problems, per the reported LHD curves (exact numbers not given in the text).
- The local kernel gives consistently lower LHD and lower variance than SVH-PSL without it on RE37, RE41 and RE42.
- Increasing the repulsion coefficient alpha reduces variance and helps particles escape pseudo-optima; SVH-PSL remains effective on high-dimensional ZDT1 and ZDT2 but with higher variance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVH-PSL instead of PSL-MOBO | Pareto set learning for expensive multi-objective optimization · Synthetic benchmarks | LHD | Helped |
| Use SVH-PSL instead of PSL-MOBO | Pareto set learning for expensive multi-objective optimization · Synthetic and real-world benchmarks | convergence speed | Helped |
| Use SVH-PSL instead of DA-PSL | Pareto set learning for expensive multi-objective optimization · Synthetic and real-world benchmarks | Pareto set diversity | Helped |
| Add the local kernel (ablation) | SVH-PSL · RE37, RE41, and RE42 | LHD | Helped |
| Add the local kernel (ablation) | SVH-PSL · RE37, RE41, and RE42 | variance | Helped |
| Increase the repulsion coefficient α (ablation) | SVH-PSL · ZDT1 | exploration toward the optimal solution | Helped |
| Use a small repulsion coefficient α (ablation) | SVH-PSL · ZDT1 | variance | Hurt |
| Increase problem dimensionality (ablation) | SVH-PSL · High-dimensional ZDT1 and ZDT2 | variance | Hurt |
Surprising findings
- Small repulsion coefficient alpha yields high variance and instability, indicating insufficient repulsion hampers escape from pseudo-optima.
Limitations
- Higher variance in high-dimensional settings.
- Relies on Gaussian process surrogates, whose fragmentation motivated the method.
- Results reported as figures; no exact numeric tables in the text.
Experimental setup
Datasets: ZDT1 ZDT2 ZDT3 ZDT4 ZDT6 F2 VLMOP2 RE21 RE32 RE33 RE36 RE37
Benchmarks: ZDT1 ZDT2 ZDT3 ZDT4 ZDT6 F2 VLMOP2 RE21 RE32 RE33 RE36 RE37
Metrics: log hypervolume difference (LHD)
Baselines: TS-TCH USeMO-EI MOEA/D-EGO TSEMO DGEMO qParEGO qEHVI qNEHVI DA-PSL PSL-MOBO
Builds on: Pareto set learning PSL-MOBO Stein Variational Gradient Descent MOO-SVGD MK-SVGD
Closest papers by meaning
- IOP: An Idempotent-Like Optimization Method on the Pareto Front of Hypernetwork (AAAI 2025 · similarity 0.88)
- Are You Concerned about Limited Function Evaluations: Data-Augmented Pareto Set Learning for Expensive Multi-Objective Optimization (AAAI 2024 · similarity 0.87)
- Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning (AAAI 2024 · similarity 0.87)
- PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization (AAAI 2025 · similarity 0.86)
- Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models (AAAI 2025 · similarity 0.86)
- Pareto Set Learning for Multi-Objective Reinforcement Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.