Research map / Probabilistic and scientific modeling / Bayesian optimization
Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CDM-PSL, a composite (conditional + unconditional) diffusion-model Pareto set learning method for expensive multi-objective Bayesian optimization, using entropy-weighted surrogate gradients to guide sampling.
Problem
Pareto set learning in expensive multi-objective Bayesian optimization is unstable with limited function evaluations, making it hard to model complex Pareto set distributions from few samples.
Approach
Replace the Pareto set learning model with a composite diffusion model: an unconditional diffusion model for diverse offspring and a conditional diffusion model whose denoising is guided by entropy-weighted gradients of GP surrogates, combined with HV-based operator switching and batch selection.
Key results
- On ZDT1 at 25 FEs, CDM-PSL reaches HV 5.6570 versus 4.9542 for the w/o Weight variant and 3.6577 for the w/o DM variant (GA-based offspring).
- Ablations at 25 FEs across six instances show each component (entropy weighting, conditional generation, diffusion model) improves HV, with removing the DM causing the largest drop (e.g., 28% to 45% gap).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add entropy-weighted gradients instead of mean-weighted gradients (ablation) | CDM-PSL · ZDT1, ZDT2, ZDT3, DTLZ3, DTLZ6, DTLZ7 | HV: 4.9542e+00 -> 5.6570e+00; 5.4062e+00 -> 6.1533e+00; 4.2659e+00 -> 5.1703e+00; 1.0391e+10 -> 1.0795e+10; 4.3509e+03 -> 4. | Helped |
| add conditional generation (ablation) | CDM-PSL · ZDT1, ZDT2, ZDT3, DTLZ3, DTLZ6, DTLZ7 | HV: 4.9587e+00 -> 5.6570e+00; 5.1008e+00 -> 6.1533e+00; 4.1583e+00 -> 5.1703e+00; 1.0420e+10 -> 1.0795e+10; 4.2265e+03 -> 4. | Helped |
| use the diffusion-model-based Pareto set learner instead of GA with SBX (ablation) | CDM-PSL · ZDT1, ZDT2, ZDT3, DTLZ3, DTLZ6, DTLZ7 | HV: 3.6577e+00 -> 5.6570e+00; 3.7327e+00 -> 6.1533e+00; 3.7185e+00 -> 5.1703e+00; 9.6465e+09 -> 1.0795e+10; 2.2088e+03 -> 4. | Helped |
| add the operator-switching strategy (ablation) | CDM-PSL · ZDT3 and DTLZ3 | HV convergence performance | Helped |
| vary the diffusion step count from the default t=25 (ablation) | CDM-PSL · synthetic benchmarks | HV convergence performance | Mixed |
| vary the learning rate from the default lr=0.001 (ablation) | CDM-PSL · d=20 benchmarks | HV | No clear effect |
| increase the conditional-generation proportion above the default CG-to-UG ratio of 1 to 10 (ablation) | CDM-PSL · d=20 benchmarks | HV | Mixed |
Surprising findings
- Performance degrades on multimodal problems (ZDT3, DTLZ3, DTLZ7) at 50 decision variables, where the method struggles with exploration and exploitation.
Limitations
- High computational overhead makes it hard to apply to very high-dimensional problems.
- Performance degrades on multimodal problems (ZDT3, DTLZ3, DTLZ7) with 50 decision variables.
Experimental setup
Datasets: ZDT1 ZDT2 ZDT3 DTLZ2 DTLZ3 DTLZ4 DTLZ5 DTLZ6 DTLZ7 RE1-RE7 real-world engineering problems
Benchmarks: ZDT1 ZDT2 ZDT3 DTLZ2 DTLZ3 DTLZ4 DTLZ5 DTLZ6 DTLZ7 RE1 RE2 RE3
Metrics: hypervolume gap
Baselines: NSGA-II MOEA/D-EGO TSEMO USeMO-EI DGEMO PSL-MOBO qNParEGO qEHVI qNEHVI
Builds on: PSL-MOBO DDOM EmoDM shift-based density estimation
Closest papers by meaning
- Are You Concerned about Limited Function Evaluations: Data-Augmented Pareto Set Learning for Expensive Multi-Objective Optimization (AAAI 2024 · similarity 0.88)
- BOtied: Multi-objective Bayesian optimization with tied multivariate ranks (ICML 2024 · similarity 0.87)
- MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions (NeurIPS 2025 · similarity 0.87)
- Preference-Aware Constrained Multi-Objective Bayesian Optimization (Student Abstract) (AAAI 2024 · similarity 0.86)
- Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks (AAAI 2025 · similarity 0.86)
- Preference-Guided Diffusion for Multi-Objective Offline Optimization (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.