Research map / Learning theory and algorithms / Prediction and decision losses
Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Geometry-Calibrated DRO (GCDRO) for regression, adding graph-based interaction and entropy calibration terms over a kNN data graph inside a geometric Wasserstein DRO set, linking the risk to Helmholtz free energy and optimizing via discretized gradient flow.
Problem
DRO suffers from over-pessimism: its worst-case distribution puts excessive weight on noisy samples with high prediction error, causing unstable parameter estimates and poor generalization under sub-population shift plus label noise.
Approach
Add a graph total-variation calibration term (sum over kNN edges of w_ij q_i q_j (l_i-l_j)^2) and a negative-entropy term to the DRO objective over a geometric Wasserstein set, interpreting the objective as Helmholtz free energy; the interaction term repels mass from locally non-smooth high-loss points, and the inner max is solved by time-discretized gradient flow with an exponential error bound.
Key results
- On simulation with strong label noise (5%), GCDRO achieves test mean RMSE 0.540 versus 0.677 for GDRO and 0.599 for ERM, with lowest test std 0.044.
- On simulation with weak label noise (0.5%), GCDRO attains the lowest parameter estimation error 0.190 versus 0.198 for GDRO.
- The paper reports GCDRO outperforms all baselines under strong distribution shifts on real-world datasets, though specific numbers are given only in figures.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GCDRO instead of DORO | Linear model · Simulation, weak label noise (0.5%) | Test Mean RMSE: 0.565 -> 0.482 | Helped |
| Use GCDRO instead of DORO | Linear model · Simulation, weak label noise (0.5%) | Test Std RMSE: 0.230 -> 0.070 | Helped |
| Use GCDRO instead of GDRO | Linear model · Simulation, weak label noise (0.5%) | Parameter estimation error: 0.198 -> 0.190 | Helped |
| Use GCDRO instead of DORO | Linear model · Simulation, strong label noise (5%) | Test Mean RMSE: 0.611 -> 0.540 | Helped |
| Use GCDRO instead of GDRO | Linear model · Simulation, strong label noise (5%) | Test Std RMSE: 0.063 -> 0.044 | Helped |
| Use GCDRO instead of GDRO | Linear model · Simulation, strong label noise (5%) | Parameter estimation error: 0.421 -> 0.268 | Helped |
| Add GCDRO calibration terms (ablation) | GDRO · Simulation with label noise | Test RMSE | Helped |
Surprising findings
- KL-DRO, chi2-DRO and GDRO concentrate weight on noisy samples, inflating effective noise roughly 10 to 15 times relative to the original level.
- DORO, which discards top-loss samples, performs poorly under strong noise because noisy points do not always have the largest prediction errors.
Limitations
- Theoretical noise-mitigation analysis is for final gradient-flow state and a simple linear example; intermediate states are not analyzed.
- Requires a kNN graph of the data manifold and grid search over alpha, beta, steps and k; model selection without domain information is hard.
- Experiments limited to regression with small MLP/linear models; classification and high-dimensional data left to future work.
Experimental setup
Datasets: Simulated regression data with sub-population shift and label noise Bike-sharing dataset House Price (King County) dataset Temperature dataset (South Korea, 2013-2017)
Benchmarks: Simulation regression (RMSE, parameter estimation error) Bike-sharing House Price Temperature
Metrics: RMSE test mean RMSE test std parameter estimation error
Baselines: ERM WDRO KL-DRO chi2-DRO GDRO HRDRO DORO
Builds on: GDRO (geometric Wasserstein DRO) KL-DRO chi-square DRO DORO HRDRO
Closest papers by meaning
- Distributionally Robust Optimization with Data Geometry (NeurIPS 2022 · similarity 0.91)
- Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization (AAAI 2024 · similarity 0.86)
- Structure-informed Risk Minimization for Robust Ensemble Learning (ICML 2025 · similarity 0.86)
- Efficient Stochastic Approximation of Minimax Excess Risk Optimization (ICML 2024 · similarity 0.85)
- Wasserstein Distributionally Robust Optimization through the Lens of Structural Causal Models and Individual Fairness (NeurIPS 2024 · similarity 0.85)
- Differentiable Distributionally Robust Optimization Layers (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.