Research map / Learning theory and algorithms / Prediction and decision losses
Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a distributionally robust criterion that averages a smooth-ambiguity-averse transform of risk over a Dirichlet process posterior, linking it to Ridge/LASSO regularization and giving finite-sample and asymptotic guarantees plus stick-breaking Monte Carlo approximations.
Problem
Empirical risk minimization can perform poorly and unstably out-of-sample under distributional uncertainty about the data-generating process, especially with small samples.
Approach
Combine a Dirichlet process posterior over the data-generating distribution with the smooth ambiguity-averse (SmAA) model, optimizing the expectation over the DP posterior of a convex increasing transform phi of the risk; this yields Ridge/LASSO-like regularization and tractable stick-breaking/Bayesian-bootstrap Monte Carlo approximations.
Key results
- Proposition shows the DP-based criterion with Gaussian base measure reduces to Ridge (phi-exponential, alpha/n penalty) and to LASSO with a suitably chosen diagonal covariance.
- Experiments report that the robust method outperforms OLS, LASSO, and unregularized logistic regression on average and yields lower performance variability across simulations and real datasets.
- Finite-sample and asymptotic theorems show excess risk vanishes almost surely as n grows with vanishing ambiguity aversion (phi_n to identity).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed robust DP criterion instead of ambiguity-neutral regularization or the unregularized procedure | Sparse linear regression, Gaussian mean estimation with outliers, and sparse logistic regression · Simulated learning tasks | out-of-sample performance | Helped |
| Use the proposed robust DP criterion instead of ambiguity-neutral regularization or the unregularized procedure | Sparse linear regression, Gaussian mean estimation with outliers, and sparse logistic regression · Simulated learning tasks | performance variability | Helped |
| Use the proposed robust DP criterion instead of L1-regularized or unregularized logistic regression | Logistic regression · Pima Indian Diabetes | out-of-sample performance | Helped |
| Use the proposed robust DP criterion instead of L1-regularized or unregularized logistic regression | Logistic regression · Pima Indian Diabetes | performance variability | Helped |
| Use the proposed robust DP criterion instead of OLS or LASSO | Linear regression · Wine Quality | out-of-sample performance | Helped |
| Use the proposed robust DP criterion instead of OLS or LASSO | Linear regression · Wine Quality | performance variability | Helped |
| Use the proposed robust DP criterion instead of OLS or LASSO | Linear regression · Liver Disorders | out-of-sample performance | Helped |
| Use the proposed robust DP criterion instead of OLS or LASSO | Linear regression · Liver Disorders | performance variability | Helped |
Limitations
- Method only suited to iid or exchangeable data
- Limited examination of parameter configuration and extension to general (e.g., deep learning) losses
Experimental setup
Datasets: Pima Indian Diabetes Wine Quality (UCI) Liver Disorders (UCI) simulated linear regression, Gaussian mean, and logistic regression data
Benchmarks: Pima Indian Diabetes Wine Quality Liver Disorders simulated sparse regression, outlier mean estimation, sparse logistic regression
Metrics: out-of-sample performance performance variability
Baselines: unregularized ERM / OLS Ridge regression LASSO L1-regularized logistic regression
Builds on: Dirichlet process smooth ambiguity aversion model Wasserstein distributionally robust optimization Bayesian bootstrap
Closest papers by meaning
- Distributionally Robust Performative Optimization (NeurIPS 2025 · similarity 0.89)
- Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.89)
- Ensemble Distributionally Robust Bayesian Optimisation (NeurIPS 2026 · similarity 0.89)
- Statistical Properties of Robust Satisficing (ICML 2024 · similarity 0.88)
- Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation (AAAI 2024 · similarity 0.88)
- Kernel-based guarantees for nonlinear parametric models in Bayesian optimization (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.