Research map / Label-efficient and robust learning / Recognition and long-tail
Knowledge-Guided Wasserstein Distributionally Robust Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes knowledge-guided Wasserstein DRO (KG-WDRO), which adds prior-knowledge-aware transport costs that restrict perturbations along prior predictor directions, showing equivalence to shrinkage-based transfer learning and improved small-sample performance in simulations.
Problem
Standard Wasserstein DRO is overly conservative, especially in small-sample regimes, shrinking estimates toward zero; the paper asks how to integrate external prior knowledge (predictors from source data) into WDRO for transfer learning.
Approach
Replace the standard transport cost with knowledge-guided cost functions that penalize (or forbid, in the strong case) perturbations of covariates along prior predictor directions, so the ambiguity set shrinks along informed directions; via strong duality and Toland's duality this yields tractable reformulations equivalent to collinearity-based shrinkage (ridge/lasso-type penalties toward the span
Key results
- In logistic simulation with N=20, s=1, rho=0.95, KG-WDRO reaches 0.817 accuracy vs 0.800 for Trans-GLM and 0.565 for WDRO.
- In linear regression simulation with N=50, s=1, rho=0.7, KG-WDRO (Strong) attains R^2 0.740 vs 0.706 for Trans-Ridge and 0.108 for WDRO.
- KG-WDRO outperforms Trans-Lasso on multi-site simulation for rho >= 0.75 in several settings, e.g. 0.916 vs 0.815 at rho=0.95, N=50, [1,-0.5,0.2].
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KG-WDRO instead of Trans-GLM | Logistic regression · High-dimensional sparse logistic regression, s=1, N=20, ρ=0.3 | classification accuracy: 0.585 -> 0.587 | Helped |
| Use KG-WDRO instead of target-only WDRO | Logistic regression · High-dimensional sparse logistic regression, s=1, N=20, ρ=0.3 | classification accuracy: 0.565 -> 0.587 | Helped |
| Use KG-WDRO strong transferring instead of Trans-Ridge | Linear regression · High-dimensional correlated linear regression, s=1, N=50, ρ=0.3 | out-of-sample R²: 0.391 -> 0.585 | Helped |
| Use KG-WDRO strong transferring instead of weak transferring (ablation) | KG-WDRO · High-dimensional correlated linear regression, s=1, N=50, ρ=0.3 | out-of-sample R²: 0.583 -> 0.585 | No clear effect |
| Use KG-WDRO instead of Trans-Lasso with unequal source weights | Multi-site linear regression · [a,b,c]=[1,-0.5,0.2], ϱ=0.9, N=50, ρ=0.7 | out-of-sample R²: 0.578 -> 0.560 | Hurt |
| Use KG-WDRO instead of Trans-Lasso with equal source weights | Multi-site linear regression · [a,b,c]=[1,1,1], ϱ=0.6, N=50, ρ=0.7 | out-of-sample R²: 0.628 -> 0.565 | Hurt |
| Use KG-WDRO instead of Trans-Lasso across multi-site settings | Multi-site linear regression · Multi-site linear regression | out-of-sample R² | Mixed |
Surprising findings
- Vanilla WDRO performs very poorly in the regression simulations (R^2 as low as 0.030 or negative), showing the conservativeness problem in small samples.
- Trans-Lasso slightly outperforms KG-WDRO at low correlation in the equal-weight multi-site setting.
Limitations
- No statistical guarantees for the proposed estimators are provided yet.
- Hyperparameters delta and lambda are selected by grid-search cross-validation without a principled rule.
- Experiments are limited to simulated data with linear regression and binary classification.
Experimental setup
Datasets: simulated data
Benchmarks: simulated logistic regression simulated linear regression simulated multi-site regression
Metrics: out-of-sample classification accuracy out-of-sample R^2
Baselines: WDRO Trans-GLM Trans-Ridge Trans-Lasso
Builds on: Wasserstein DRO strong duality Trans-GLM Trans-Lasso Trans-Ridge Toland's duality
Closest papers by meaning
- Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization (ICLR 2025 · similarity 0.89)
- Universal generalization guarantees for Wasserstein distributionally robust models (ICLR 2025 · similarity 0.87)
- Wasserstein Transfer Learning (NeurIPS 2025 · similarity 0.86)
- Wasserstein Distributionally Robust Optimization through the Lens of Structural Causal Models and Individual Fairness (NeurIPS 2024 · similarity 0.86)
- Adversarially Robust Hypothesis Transfer Learning (ICML 2024 · similarity 0.85)
- Group Distributionally Robust Optimization with Flexible Sample Queries (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.