Research map / Learning theory and algorithms / Combinatorial and search optimization
Many-Objective Multi-Solution Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MosT, a bi-level optimal-transport-weighted multi-gradient framework that trains m solutions to cover n >> m objectives (clients, tasks, instances) with diverse Pareto-stationary solutions.
Problem
Finding m diverse Pareto-stationary solutions (models) that jointly cover many objectives (n >> m), where standard MOO methods such as MGDA and linearization either scale poorly or leave objectives ignored.
Approach
Model the objective-to-solution weight matrix as an optimal transport plan with marginal constraints (complementary coverage of objectives and balanced load on solutions) plus a diversity regularizer, and alternate OT matching with a Gamma-reweighted MGDA update for each solution; an extension (MosT-E) adds Dirichlet-interpolated objectives for n << m.
Key results
- On FEMNIST, MosT reaches 81.16% mean client accuracy vs 80.08% for FedMGDA+ and 72.47% for FedAvg (n=206 clients, m solutions).
- On Office-Caltech10 multi-task learning, MosT achieves 82.98% average accuracy vs 80.74% for MGDA and 61.26% for Linearization.
- On ZDT-1, MosT achieves hypervolume 5.87 vs 5.72 for Linearization and 4.40 for EPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MosT instead of the strongest listed baseline (MGDA) | Federated learning · Syn (0.0, 0.0) | accuracy: 77.22$_{\pm 0.41}$ -> 84.25$_{\pm 0.51}$ | Helped |
| Remove diversity regularization R(Γ) (ablation) | MosT · Syn (0.0, 0.0) | accuracy: 84.25$_{\pm 0.51}$ -> 83.09$_{\pm 0.87}$ | Hurt |
| Remove diversity regularization R(Γ) (ablation) | MosT · Syn (0.5, 0.5) | accuracy: 89.99$_{\pm 0.52}$ -> 89.07$_{\pm 0.63}$ | Hurt |
| Remove diversity regularization R(Γ) (ablation) | MosT · Syn (1.0, 1.0) | accuracy: 92.21$_{\pm 0.08}$ -> 91.70$_{\pm 0.02}$ | Hurt |
| Remove diversity regularization R(Γ) (ablation) | MosT · FEMNIST | accuracy: 81.16$_{\pm 0.03}$ -> 80.94$_{\pm 0.34}$ | Hurt |
| Use MosT instead of the strongest listed baseline (MGDA) | Multi-task learning · Office-10 | accuracy: 80.74$_{\pm 0.44}$ -> 82.98$_{\pm 0.51}$ | Helped |
| Use MosT instead of the strongest listed baseline (MGDA) | Multi-task learning · DomainNet | accuracy: 65.81$_{\pm 0.37}$ -> 67.65$_{\pm 0.55}$ | Helped |
| Use MosT instead of the strongest listed baseline (MGDA) | Mixture-of-prompt learning · BoolQ | accuracy: 62.69$_{\pm 0.71}$ -> 67.03$_{\pm 0.49}$ | Helped |
| Use MosT instead of the strongest listed baseline (MGDA) | Mixture-of-prompt learning · MultiRC | accuracy: 60.86$_{\pm 0.50}$ -> 63.78$_{\pm 0.15}$ | Helped |
| Use MosT instead of the strongest listed baseline (Linearization) | Mixture-of-prompt learning · WiC | accuracy: 57.16$_{\pm 1.27}$ -> 62.38$_{\pm 0.31}$ | Helped |
| Use MosT instead of the strongest listed baseline (Linearization) | Multi-solution MOO · ZDT-1 | Hypervolume: 5.72$_{\pm 0.01}$ -> 5.87$_{\pm 0.00}$ | Helped |
| Use MosT-E objective interpolation instead of MosT (ablation) | Fairness-accuracy trade-offs · Synthetic | Hypervolume (×100): 3.78$_{\pm 0.04}$ -> 7.65$_{\pm 0.06}$ | Helped |
Surprising findings
- Linearization-based MOO and the MGDA baseline show large gaps versus MosT on multi-task learning (e.g., 61.26% vs 82.98% on Office-Caltech10), and MGDA alone is strong on FEMNIST relative to FedAvg.
- EPO with 100 preference vectors still underperforms MosT with only 5 solutions on ZDT problems.
Limitations
- Diversity regularization and curriculum add hyperparameters (tau, alpha, beta schedules) that require tuning.
- Convergence guarantees are shown under smoothness and full-gradient assumptions; practical mini-batch implementation relies on heuristics.
- For n << m, the basic formulation can collapse to duplicate solutions, requiring the MosT-E extension.
Experimental setup
Datasets: Synthetic federated data FEMNIST Office-Caltech10 DomainNet SuperGLUE (BoolQ, MultiRC, WiC) ZDT-1/2/3 German credit synthetic fairness dataset
Benchmarks: FEMNIST Synthetic federated data Office-Caltech10 DomainNet BoolQ MultiRC WiC ZDT-1 ZDT-2 ZDT-3 German credit
Metrics: mean accuracy across clients worst-client tail accuracy hypervolume test accuracy runtime
Baselines: MGDA Linearization FedAvg FedProx FedMGDA+ EPO SVGD
Builds on: MGDA optimal transport (IPOT) EPO FedMGDA+
Closest papers by meaning
- Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimization (ICLR 2025 · similarity 0.86)
- Voronoi-grid-based Pareto Front Learning and Its Application to Collaborative Federated Learning (ICML 2025 · similarity 0.85)
- Preference-Guided Diffusion for Multi-Objective Offline Optimization (NeurIPS 2025 · similarity 0.85)
- IOP: An Idempotent-Like Optimization Method on the Pareto Front of Hypernetwork (AAAI 2025 · similarity 0.85)
- ParetoFlow: Guided Flows in Multi-Objective Optimization (ICLR 2025 · similarity 0.84)
- Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.