Research map / Optimization / Optimizers
LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LibMOON, an open-source PyTorch library of gradient-based multiobjective optimization solvers (finite-solution MOO, Pareto set learning, and MOBO) with benchmarks on synthetic, MTL fairness/MO-MNIST, and real-world problems.
Problem
Existing multiobjective optimization libraries focus on zeroth-order evolutionary algorithms that do not use gradient information and do not scale to models with millions of parameters.
Approach
A modular, GPU-accelerated PyTorch library that unifies over twenty gradient-based MOO methods (aggregation, gradient-manipulation, Pareto set learning, decomposition-based MOBO) under one benchmark with fair comparisons on synthetic, multitask, and real-world problems.
Key results
- On VLMOP2, Agg-Tche was recommended as most reliable finite-solution solver for exact Pareto solutions, while HVGrad attained the highest hypervolume (0.286).
- On Adult fairness MTL, HVGrad achieved the best HV (0.678), with Agg-mTche and Agg-SmoothmTche also performing well.
- GPU acceleration on MO-MNIST reduced running time from 43.43 min on CPU to 3.27 min on RTX 4090 for 100 epochs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HVGrad instead of other finite Pareto solution solvers | Finite Pareto solution optimization · VLMOP2 | HV: 0.283 (0.000) -> 0.286 (0.000) | Helped |
| Use Agg-Tche, EPO, or PMGDA as the PSL base solver rather than COSMOS or Agg-LS | Pareto set learning · VLMOP2 | HV: 0.318 (0.000) -> 0.319 (0.000) | Helped |
| Use Agg-Tche instead of Agg-LS for finite Pareto solutions | Finite Pareto solution optimization · VLMOP2 | HV: 0.043 (0.002) -> 0.283 (0.000) | Helped |
| Use HVGrad instead of Agg-LS for finite Pareto solutions | Finite Pareto solution optimization · Adult | HV: 0.636 (0.001) -> 0.678 (0.001) | Helped |
| Use Agg-LS-based PSL rather than other reported PSL methods | Pareto set learning · MO-MNIST | HV: 0.512 (0.015) -> 0.557 (0.004) | Helped |
| Run PSL on an RTX 4060 instead of a CPU | LibMOON PSL · MO-MNIST | running time (minutes): 43.43 -> 14.72 | Helped |
| Use an RTX 4090 instead of an RTX 4060 | LibMOON PSL · MO-MNIST | running time (minutes): 14.72 -> 3.27 | Helped |
| Use Agg-LS for Pareto set learning | Pareto set learning · MO-MNIST | training loss | No clear effect |
| Use COSMOS-based PSL instead of PSL methods that recover the full front | Pareto set learning · VLMOP2 | Pareto-front coverage | Hurt |
| Use linear-scalarization PSL instead of PSL methods that recover the full front | Pareto set learning · VLMOP2 | Pareto-front coverage | Hurt |
Surprising findings
- Agg-LS fails to find non-convex parts of the Pareto front and finds only two endpoints on VLMOP2.
- MOO-SVGD and Agg-PBI show large run-to-run variance and instability.
Limitations
- Rapid development of the field means some effective methods may be missing.
- Gradient-based solvers may get stuck in local optima on problems with many local optima.
Experimental setup
Datasets: VLMOP2 VLMOP1 ZDT1 RE21 Adult Compas Credit MO-MNIST Fashion-MNIST
Benchmarks: VLMOP2 VLMOP1 ZDT1 RE21 Adult MO-MNIST
Metrics: hypervolume IGD minimal distance spacing sparsity span inner product cross angle PBI running time
Baselines: Agg-LS Agg-Tche Agg-PBI Agg-COSMOS MGDA-UB Random MOO-SVGD PMTL HVGrad PMGDA EPO
Builds on: EPO MGDA-UB PMTL MOO-SVGD HVGrad COSMOS Pareto hypernetworks
Closest papers by meaning
- ParetoFlow: Guided Flows in Multi-Objective Optimization (ICLR 2025 · similarity 0.85)
- Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization (AAAI 2024 · similarity 0.84)
- Offline Multi-Objective Optimization (ICML 2024 · similarity 0.83)
- MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions (NeurIPS 2025 · similarity 0.83)
- PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization (AAAI 2025 · similarity 0.83)
- BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.