Research map / Learning theory and algorithms / Optimal transport
Outlier-Robust Distributionally Robust Optimization via Unbalanced Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an unbalanced-optimal-transport (UOT) distance-based DRO framework that softly penalizes marginal mismatch to be robust to outliers, proves strong duality, and gives a Lagrangian penalty form solved by stochastic subgradient descent.
Problem
Distributionally robust optimization degrades when the empirical distribution is contaminated by outliers, since Wasserstein ambiguity sets must be made very large to include the clean distribution, yielding overly conservative solutions.
Approach
Replace hard marginal constraints of Wasserstein DRO with a soft KL penalty on the intermediate marginal (unbalanced OT distance), and combine with a prior-knowledge outlier-penalty function h so outlier-heavy distributions are down-weighted; the Lagrangian penalty dual becomes an exponentially reweighted empirical loss solvable by stochastic subgradient.
Key results
- On linear regression with n=20000, UOT-DRO runs in 22.2 s versus over 12000 s (timeout) for OR-WDRO, with excess risk 0.031.
- In linear classification at contamination C=6, UOT-DRO reaches 92% accuracy versus 73% for standard DRO and 72% for OR-WDRO.
- In logistic regression at contamination C=16, UOT-DRO keeps about 92% accuracy versus 56% for standard DRO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UOT-DRO instead of standard DRO | Linear classification · Linear classification, contamination C=6 | excess risk: 0.628 -> 0.298 | Helped |
| Use UOT-DRO instead of OR-WDRO | Linear classification · Linear classification, contamination C=10 | excess risk: 0.560 -> 0.295 | Helped |
| Use UOT-DRO instead of OR-WDRO | Linear classification · Linear classification, contamination C=30 | accuracy: 90% -> 96% | Helped |
| Use UOT-DRO instead of OR-WDRO | Linear classification · Linear classification, contamination C=100 | accuracy: 97% -> 95% | Hurt |
| Use UOT-DRO instead of standard DRO | Logistic regression · Logistic regression, contamination C=16 | accuracy: 56% -> 92% | Helped |
| Use UOT-DRO instead of OR-WDRO | Linear regression · Linear regression, n=20000 | running time: * -> 22.2 | Helped |
| Remove the prior-knowledge penalty (set λ2 = 0) (ablation) | UOT-DRO · Linear regression | excess risk: 0.057 -> 4.914 | Hurt |
| Add prior knowledge with λ2 = 1 instead of λ2 = 0 (ablation) | UOT-DRO · Linear regression | excess risk: 4.914 -> 1.340 | Helped |
| Increase the prior-knowledge weight from λ2 = 1 to λ2 = 2 (ablation) | UOT-DRO · Linear regression | excess risk: 1.340 -> 0.061 | Helped |
| Increase the prior-knowledge weight from λ2 = 4 to λ2 = 8 (ablation) | UOT-DRO · Linear regression | excess risk: 0.059 -> 0.063 | Hurt |
| Vary the preprocessing contamination estimate ˆε (ablation) | UOT-DRO · Linear regression | excess risk: 0.049; 0.049; 0.050 | No clear effect |
| Vary the preprocessing contamination estimate ˆε (ablation) | UOT-DRO · Linear classification | excess risk: 0.283; 0.283; 0.282 | No clear effect |
Surprising findings
- Using the UOT distance without prior knowledge (lambda2=0) performs worse than standard DRO in linear regression, since the ambiguity set then includes outlier-heavy distributions.
Limitations
- Strong duality for the primal requires smoothness assumptions and a strictly positive optimal dual variable.
- Performance depends on the prior-knowledge outlier function h and its credibility parameter lambda2; without prior knowledge the method can be worse than standard DRO.
- Experiments are limited to small synthetic settings with linear models.
Experimental setup
Datasets: synthetic Gaussian regression data synthetic Gaussian classification data synthetic logistic regression data
Benchmarks: synthetic linear regression synthetic linear classification synthetic logistic regression
Metrics: excess risk accuracy running time
Baselines: Standard DRO OR-WDRO standard DRO (logistic regression, Shafieezadeh-Abadeh et al.)
Builds on: Wasserstein DRO Lagrangian penalty DRO (Gao et al.) outlier-robust WDRO (Nguyen et al.) unbalanced optimal transport
Closest papers by meaning
- Robust Learning via Nested Distributionally Robust Optimization (ICML 2026 · similarity 0.91)
- Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization (AAAI 2024 · similarity 0.89)
- Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.89)
- Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.89)
- Designing Ambiguity Sets for Distributionally Robust Optimization Using Structural Causal Optimal Transport (AAAI 2025 · similarity 0.88)
- Wasserstein Distributionally Robust Optimization through the Lens of Structural Causal Models and Individual Fairness (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.