Research map / Learning theory and algorithms / Prediction and decision losses
Criterion Collapse and Loss Distribution Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies "criterion collapse" where optimizing risk criteria (DRO, OCE/CVaR, tilted ERM, quantiles, non-monotonic variantile-style and loss-restraining criteria) under Bernoulli zero-one loss or surrogate losses yields error-probability minimizers, and tests Flooding and SoftAD against monotonic criteria on image classification.
Problem
When does optimizing a learning criterion other than the expected loss (e.g., CVaR, DRO, tilted ERM) actually change the learned solution, versus collapsing into error probability minimizers, under zero-one and surrogate losses?
Approach
Shows that for Bernoulli (zero-one) losses, monotonic criteria (OCE, DRO, quantiles, tilted risk) always include error-probability minimizers in their solution set; with surrogate margin losses and a highly expressive model, monotonic OCE criteria cannot avoid collapse, whereas non-monotonic loss-restraining criteria (inner/outer, realized by Flooding and SoftAD) can.
Key results
- Under the zero-one loss, monotonic criteria such as DRO, CVaR, quantiles and tilted risk have error-probability minimizers included in their solution sets, and tilted risk solution sets coincide exactly with error minimizers.
- In experiments over 5 trials, validation-accuracy-selected non-monotonic methods (Flooding, SoftAD) achieved better average test surrogate loss than monotonic methods, with SoftAD also giving a small model norm.
- Tilted ERM was selected at gamma=0 (vanilla ERM) for all datasets and trials, and all methods reached near-perfect training accuracy with similar test accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Flooding or SoftAD instead of monotonic criteria | Image classification with cross-entropy training · CIFAR-10, CIFAR-100, FashionMNIST, SVHN test sets | average surrogate loss | Helped |
| Use SoftAD instead of the other tested criteria | Image classification with cross-entropy training · CIFAR-10, CIFAR-100, FashionMNIST, SVHN | model L2 norm | Helped |
| Select each method's hyperparameter to maximize validation accuracy | CVaR, DRO, Flooding, SoftAD, and tilted ERM · CIFAR-10, CIFAR-100, FashionMNIST, SVHN | test accuracy | No clear effect |
| Tune positive-tilt γ for validation accuracy | Tilted ERM · CIFAR-10, CIFAR-100, FashionMNIST, SVHN | validation accuracy | No clear effect |
Surprising findings
- Tilted ERM and DRO selection on validation accuracy reduced to vanilla ERM on several datasets, indicating no accuracy benefit from monotonic criteria.
- Non-monotonic criteria offer better test surrogate loss and model norm despite equal test accuracy.
Limitations
- Main results cover only margin-type losses for binary classification; asymmetric and multi-class surrogates left to future work.
- Unavoidability results assume a highly expressive model class; with limited capacity collapse need not occur.
- Experiments only measure accuracy, surrogate loss, and norm; fairness and calibration not evaluated.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 FashionMNIST SVHN
Benchmarks: CIFAR-10 CIFAR-100 FashionMNIST SVHN
Metrics: accuracy average surrogate loss L2 norm of model parameters
Baselines: ERM CVaR DRO Tilted ERM Flooding SoftAD
Builds on: CVaR DRO Flooding SoftAD tilted ERM
Closest papers by meaning
- From Non-Convex to Strongly Convex: Curvature-Adaptive FTPL for Online Optimization (NeurIPS 2026 · similarity 0.84)
- A Reduction from Delayed to Immediate Feedback for Online Convex Optimization with Improved Guarantees (NeurIPS 2026 · similarity 0.84)
- Establishing Linear Surrogate Regret Bounds for Convex Smooth Losses via Convolutional Fenchel–Young Losses (NeurIPS 2025 · similarity 0.84)
- Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.84)
- Exploiting Curvature in Online Convex Optimization with Delayed Feedback (ICML 2025 · similarity 0.84)
- Stochastic Optimization Schemes for Performative Prediction with Nonconvex Loss (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.