Research map / Label-efficient and robust learning / Domain adaptation and generalization
RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoME, a mixture-of-experts GNN with two-level distributionally robust training that predicts MILP solutions across domains and generalizes zero-shot to unseen domains and MIPLIB.
Problem
Learning-based MILP solution prediction is usually trained and evaluated in single domains, so models overfit and fail to generalize to unseen problem distributions.
Approach
Combine a mixture-of-experts GNN (graph-embedding-based gating for encoder and decoder) with two-level DRO: group-DRO across domains and isotropic perturbation of task embeddings with a fuzzy-membership consistency loss, plus an expert diversity regularizer.
Key results
- A single RoME model trained on IS, IP, SC with Gurobi reduces the absolute primal gap vs Gurobi by 77.8% on IP, 91.2% on SC, and 85.7% on WA (zero-shot), and 83.7% on CA (zero-shot).
- On IS, RoME+Gurobi reaches the optimum in 2.08 s versus 57.35 s for Gurobi (27.5x speedup).
- On the MIPLIB hard-instance group, RoME+Gurobi attains the best-known objective (gap 0.00) versus 325.40 for Gurobi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train one RoME model across IS, IP, and SC instead of training a model separately for each domain | RoME+Gurobi · IS, IP, SC, WA, and CA | absolute primal gap (IS: time): IS: 20.13 -> 2.08; IP: 0.20 -> 0.06; SC: 0.54 -> 0.05; WA: 0.42 -> 0.06; CA: 128.54 -> 35.09 | Helped |
| Apply RoME pretrained on synthetic domains to MIPLIB without retraining | RoME+Gurobi · MIPLIB IIS | absolute primal gap: 14.20 -> 10.80 | Helped |
| Apply RoME pretrained on synthetic domains to MIPLIB without retraining | RoME+Gurobi · MIPLIB hard instances | absolute primal gap: 325.40 -> 0.00 | Helped |
| Add DRO to the MoE model (ablation) | RoME · IP, SC, WA, and CA | absolute primal gap: IP: 0.11 -> 0.06; SC: 0.01 -> 0.05; WA: 5.18 -> 0.06; CA: 2241.86 -> 35.09 | Mixed |
| Use MoE instead of replacing it with the PS GCN (ablation) | RoME · IP, SC, WA, and CA | absolute primal gap: IP: 0.11 -> 0.06; SC: 0.00 -> 0.05; WA: 1.08 -> 0.06; CA: 329.39 -> 35.09 | Mixed |
| Add expert diversity using the multi-head design (ablation) | RoME · CA | absolute primal gap: 5276.19 -> 35.09 | Helped |
| Add task-embedding perturbation rather than using no perturbation (ablation) | RoME · CA | absolute primal gap: 2625.7 -> 35.09 | Helped |
| Perturb task embeddings instead of variable embeddings or variable features (ablation) | RoME · CA | absolute primal gap: 3221.66 -> 35.09; 6052.18 -> 35.09 | Helped |
| Vary perturbation ratio across the reported values (ablation) | RoME · CA | absolute primal gap: 0.05: 6162.07; 0.10: 35.09; 0.15: 150.05; 0.20: 4161.69 | Mixed |
| Increase expert diversity ratio from 0.1 to 0.2 (ablation) | RoME · CA | absolute primal gap: 2625.70 -> 35.09 | Helped |
Surprising findings
- Cross-domain training improves performance within individual domains compared with per-domain baselines trained on the same domain.
- The single-head variant achieves the best IP gap, slightly beating full RoME on that domain.
Limitations
- High training cost; limited by GPU memory, only 300 samples per cross-domain dataset were used.
- Evaluation relies on synthetic training data and a curated MIPLIB subset, with best-known solutions approximated by a 3600s single-thread Gurobi run for some domains.
Experimental setup
Datasets: Independent Set (IS) synthetic instances Set Covering (SC) synthetic instances Item Placement (IP) from NeurIPS 2021 ML4CO Combinatorial Auctions (CA) synthetic instances Workload Appointment (WA) from NeurIPS 2021 ML4CO MIPLIB subsets
Benchmarks: IS IP SC WA CA MIPLIB subsets (IIS, hard instances)
Metrics: best objective value absolute primal gap vs best-known solution solving time (IS) win count
Baselines: Gurobi SCIP Predict-and-Search Contrastive Predict-and-Search
Builds on: Predict-and-Search Group DRO Mixture of Experts
Closest papers by meaning
- Apollo-MILP: An Alternating Prediction-Correction Neural Solving Framework for Mixed-Integer Linear Programming (ICLR 2025 · similarity 0.89)
- Towards Foundation Models for Mixed Integer Linear Programming (ICLR 2025 · similarity 0.89)
- ACM-MILP: Adaptive Constraint Modification via Grouping and Selection for Hardness-Preserving MILP Instance Generation (ICML 2024 · similarity 0.87)
- Contrastive Predict-and-Search for Mixed Integer Linear Programs (ICML 2024 · similarity 0.87)
- Fast and Interpretable Mixed-Integer Linear Program Solving by Learning Model Reduction (AAAI 2025 · similarity 0.87)
- MILP-StuDio: MILP Instance Generation via Block Structure Decomposition (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.