Research map / Learning theory and algorithms / Prediction and decision losses
Learning with Fitzpatrick Losses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Fitzpatrick losses, a family of convex primal-dual losses built from the Fitzpatrick function that are tighter lower bounds of Fenchel-Young losses while keeping the same link function, instantiated as Fitzpatrick logistic and sparsemax losses for label proportion estimation.
Problem
Fenchel-Young losses are convex and tied to a link function, but only one loss is associated with each link; can new convex losses share the same link function (e.g., softargmax, sparsemax) and be tighter?
Approach
Replace the Fenchel-Young inequality with the refined inequality given by the Fitzpatrick function of the subdifferential, yielding losses that are always lower bounds of Fenchel-Young losses, share the same link function, remain convex in each argument, and equal Fenchel-Young losses with a target-dependent generating function.
Key results
- Fitzpatrick logistic loss and logistic loss are comparable: logistic significantly wins on 2 of 11 datasets and Fitzpatrick logistic on 2 of 11.
- Fitzpatrick sparsemax loss is a serious contender to sparsemax: sparsemax significantly wins on 1 dataset, Fitzpatrick sparsemax on 2 of 11 (e.g., Ecthr A tie at 0.514 vs 0.514; Scene 0.355 vs 0.363).
- Fitzpatrick logistic gives Ecthr A MSE 0.423 vs 0.431 for logistic; Mediamill favors standard losses (0.191 sparsemax vs 0.203 Fitzpatrick sparsemax).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · 11 label-proportion datasets | MSE: 2 datasets vs 2 datasets | Mixed |
| Replace sparsemax loss with Fitzpatrick sparsemax loss | Sparse argmax label-proportion estimator · 11 label-proportion datasets | MSE: 1 dataset vs 2 datasets | Mixed |
| Replace sparsemax loss with Fitzpatrick sparsemax loss | Sparse argmax label-proportion estimator · Birds | MSE: 0.531 -> 0.513 | Helped |
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · Ecthr A | MSE: 0.431 -> 0.423 | Helped |
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · Emotions | MSE: 0.327 -> 0.320 | Helped |
| Replace sparsemax loss with Fitzpatrick sparsemax loss | Sparse argmax label-proportion estimator · Scene | MSE: 0.363 -> 0.355 | Helped |
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · Scene | MSE: 0.344 -> 0.368 | Hurt |
| Replace sparsemax loss with Fitzpatrick sparsemax loss | Sparse argmax label-proportion estimator · Mediamill | MSE: 0.191 -> 0.203 | Hurt |
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · Mediamill | MSE: 0.207 -> 0.220 | Hurt |
| Replace logistic loss with Fitzpatrick logistic loss | Soft argmax label-proportion estimator · Delicious | MSE: 0.056 -> 0.055 | Helped |
| Replace sparsemax loss with Fitzpatrick sparsemax loss | Sparse argmax label-proportion estimator · Cal500 | MSE: 0.035 -> 0.035 | No clear effect |
Surprising findings
- Fitzpatrick losses are not uniformly better: standard Fenchel-Young losses win on several datasets despite being looser bounds.
Limitations
- Fitzpatrick logistic loss requires solving a root equation (bisection) for loss and gradient, increasing computational cost.
- Empirical evaluation is limited to linear models on label proportion estimation with MSE.
- Code not yet released at submission time.
Experimental setup
Datasets: Birds Cal500 Delicious Ecthr A Emotions Flags Mediamill Scene Tmc Unfair Yeast
Benchmarks: Birds Cal500 Delicious Ecthr A Emotions Flags Mediamill Scene Tmc Unfair Yeast
Metrics: mean squared error on label proportions
Baselines: Fenchel-Young sparsemax loss Fenchel-Young logistic loss
Builds on: Fenchel-Young losses sparseMAP Fitzpatrick function generalized Bregman divergence
Closest papers by meaning
- Loss Functions and Operators Generated by f-Divergences (ICML 2025 · similarity 0.86)
- Supervised learning: no loss no cry (ICML 2020 · similarity 0.86)
- Any-stepsize Gradient Descent for Separable Data under Fenchel–Young Losses (NeurIPS 2025 · similarity 0.85)
- Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms (NeurIPS 2024 · similarity 0.84)
- Realizable $H$-Consistent and Bayes-Consistent Loss Functions for Learning to Defer (NeurIPS 2024 · similarity 0.84)
- Establishing Linear Surrogate Regret Bounds for Convex Smooth Losses via Convolutional Fenchel–Young Losses (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.