Research map / Learning theory and algorithms / Prediction and decision losses

Structure-informed Risk Minimization for Robust Ensemble Learning

ICML 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Distributionally robust optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Structure-informed Risk Minimization (SRM), which learns OoD-robust ensemble weights by building a DRO-style uncertainty set over training distributions weighted by a graph-centrality prior on a distributional graph.

Problem

Ensemble weights tuned on validation data may not transfer to unseen OoD test distributions, while standard DRO uncertainty sets are overly pessimistic because they ignore structure among distributions.

Approach

Builds a graph over training distributions with Gaussian-approximated 2-Wasserstein edge weights, derives a closeness-centrality prior, and restricts the adversarial mixture weights to a divergence ball around this prior, so the minimax optimization over ensemble weights focuses on plausible shifts rather than arbitrary worst cases.

Key results

• On DomainBed, SRM reaches 66.5% average accuracy versus 66.2% for DRO and 66.0% for uniform ensemble, with best results on VLCS, OfficeHome, DomainNet and TerraIncognita.
• On FMoW-WILDS Test After 2016, SRM achieves 37.81% worst-region accuracy versus 36.98% for DRO.
• On FMoW-WILDS Test Before 2004, SRM achieves 54.44% worst-region accuracy versus 53.21% for DRO.

Measured effects reported in the paper

Surprising findings

• Greedy selection and uniform ensembling are competitive with optimization-based methods on DomainBed, with gains over the best baseline often under 0.5 points.

Limitations

• Gains over DRO and other baselines are small (often well under one accuracy point) on DomainBed and FMoW-WILDS.
• Performance gap to DRO shrinks as corruption severity increases, and results depend on the tuning of lambda and the graph sparsity.
• Evaluated on only two benchmarks, both image classification with pre-trained models.

Experimental setup

Datasets: DomainBed (PACS, VLCS, OfficeHome, TerraIncognita, DomainNet) FMoW-WILDS

Benchmarks: DomainBed FMoW-WILDS

Metrics: average accuracy worst-region accuracy

Baselines: Uniform Ensemble Greedy Selection ERM Uniform Prior Laplacian Prior Group DRO

Builds on: Group DRO empirical risk minimization Wasserstein distance

Closest papers by meaning

• Efficient Generalization with Distributionally Robust Learning (NeurIPS 2021 · similarity 0.89)
• Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.88)
• Distributionally Robust Performative Optimization (NeurIPS 2025 · similarity 0.87)
• MixMax: Distributional Robustness in Function Space via Optimal Data Mixtures (ICLR 2025 · similarity 0.87)
• Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization (AAAI 2024 · similarity 0.87)
• Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.86)