Research map / Learning theory and algorithms / Prediction and decision losses
Distributionally robust optimization
Train models against worst-case distributions using robust optimization formulations.
184 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Prediction and decision losses, keyword cluster “selection · robust · test”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
45 papers in 2023–24 and 46 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 8.4% to 8.0%.
Open questions the data points to
- Papers report ERM on WaterBirds test (average accuracy) anywhere from 85.2 to 97.1: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs (ICML 2016 · 42 citations)
- 2017 · Positive-Unlabeled Learning with Non-Negative Risk Estimator (NeurIPS 2017 · 136 citations)
- 2018 · Wasserstein Distributionally Robust Kalman Filtering (NeurIPS 2018 · 30 citations)
- 2019 · MAGSAC: Marginalizing Sample Consensus (CVPR 2019 · 357 citations)
- 2020 · Stable Prediction with Model Misspecification and Agnostic Distribution Shift (AAAI 2020 · 115 citations)
- 2021 · The Risks of Invariant Risk Minimization (ICLR 2021 · 88 citations)
- 2022 · A Consistent and Differentiable Lp Canonical Calibration Error Estimator (NeurIPS 2022 · 9 citations)
- 2023 · Multiple Robust Learning for Recommendation (AAAI 2023 · 24 citations)
- 2024 · The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · 14 citations)
- 2025 · Transfer Learning Meets Functional Linear Regression: No Negative Transfer Under Posterior Drift (AAAI 2025 · 2 citations)
- 2026 · Unified Minimax Optimization Framework for Propensity Score Estimation in Debiased Recommendation (AAAI 2026 · 2 citations)
Earliest papers here
- Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs (ICML 2016 · 42 citations)
- The knockoff filter for FDR control in group-sparse and multitask regression (ICML 2016 · 23 citations)
- Accumulated Stability Voting: A Robust Descriptor From Descriptors of Multiple Scales (CVPR 2016 · 22 citations)
What has been tried
149 papers here report 1,317 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 12%, hurt 17%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use Exp instead of ReLU activation in DER | SST-5 · MSE | Helped | The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · 14 citations) | |
| Vary the uncertainty regularization coefficient λ₁ | Cubic regression outside HUA · uncertainty estimation | Mixed | Uncertainty Regularized Evidential Regression (AAAI 2024 · 13 citations) | |
| remove elevation predictor | New Zealand · MSE | 63.6±2.0 -> 65.2±1.4 | Hurt | Quantile-Regression-Ensemble: A Deep Learning Algorithm for Downscaling Extreme Precipitation (AAAI 2024 · 11 citations) |
| Replace sparse aggregation with Q-aggregation | Synthetic Model II · MISE | No clear effect | Transfer Learning Meets Functional Linear Regression: No Negative Transfer Under Posterior Drift (AAAI 2025 · 2 citations) | |
| Expand the sigma search interval, holding step τ=0.4 and left endpoint at 0.12 | CIFAR-10, σ=0.5, id%50==0 · ACR | 0.619; 0.627; 0.638 | Helped | Towards Large Certified Radius in Randomized Smoothing Using Quasiconcave Optimization (AAAI 2024 · 1 citation) |
| increase slider τ from 0 to 0.9 | Normal, p=5, δ=2, n=500 · empirical power (%) | 46.9 -> 100.0 | Helped | Zipper: Addressing Degeneracy in Algorithm-Agnostic Inference (NeurIPS 2024 · 1 citation) |
| Add structural regularization with moderate λ (around 1.0) rather than λ = 0 | FMoW-WILDS Test Middle and Test After 2016 · worst-region accuracy (%) | Helped | Structure-informed Risk Minimization for Robust Ensemble Learning (ICML 2025 · 0 citations) | |
| Use the α-hat-prime weight estimator instead of α-hat | Finite-sample estimation across evaluated datasets · MSE | Mixed | A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators (ICML 2025 · 0 citations) | |
| Project influence estimates onto high-curvature eigenspaces, excluding lower-curvature directions | MNIST and FashionMNIST · LDS | Hurt | Better Training Data Attribution via Better Inverse Hessian-Vector Products (NeurIPS 2025 · 0 citations) | |
| Set VAE bottleneck dimension to L=2 rather than L=4 or L=8 | New England capacity expansion · VaR approximation error | 14% (52) vs 114% (64) and 112% (40) | Helped | A Deep Generative Learning Approach for Two-stage Adaptive Robust Optimization (ICLR 2025 · 0 citations) |
| Use vector parameterization instead of neural-network parameterization | Spambase · logistic test loss | 2.57 ± 0.19 vs 2.56 ± 0.13 | No clear effect | A Differentiable Rank-Based Objective for Better Feature Learning (ICLR 2025 · 0 citations) |
| Increase the initial regularization ρ | Baird's Example · early convergence speed | Helped | Regularized Q-learning through Robust Averaging (ICML 2024 · 0 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| ERM | WaterBirds test | average accuracy | 85.2 – 97.1 | 2 papers, e.g. ICLR 2022 and ICLR 2023 |
What reviewers will expect
Datasets and baselines named in the 26 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 5 of 26 papers
- Waterbirds · 3 of 26 papers
- CivilComments · 3 of 26 papers
- MNIST · 3 of 26 papers
- CIFAR100 · 2 of 26 papers
- UCI Concrete · 2 of 26 papers
- QSAR · 2 of 26 papers
- Metashift · 2 of 26 papers
Baselines
- ERM · 3 of 26 papers
- Random · 3 of 26 papers
- Mixup · 2 of 26 papers
- GroupDRO · 2 of 26 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 91 papers; from OpenAlex and the papers' affiliations).
- Carnegie Mellon University · 5 papers
- Peking University · 3 papers
- University of Bristol · 3 papers
- Meta · 3 papers
- Columbia University · 3 papers
- Tsinghua University · 3 papers
- University of California, Berkeley · 3 papers
- Huazhong Agricultural University · 2 papers
- Princeton University · 2 papers
- Xi’an Jiaotong University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Active learning and subset selection (199 papers)
- Adaptive testing and experiments (201 papers)
- Data attribution and recourse (160 papers)
- Theory of neural network sensitivity (143 papers)
- Determinantal point process inference (122 papers)
Elsewhere in the topic
- Ranking models from comparisons (127 papers)
- Conformal prediction sets (116 papers)
- Provable post-hoc explanations (92 papers)
- Tree ensemble optimization (77 papers)
- Optimal decision tree learning (75 papers)
- Consistency of surrogate losses (72 papers)
- Losses for decision-making (68 papers)
- Shapley value attribution (63 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).