Research map / Learning theory and algorithms / Prediction and decision losses
Interplay of ROC and Precision-Recall AUCs: Theoretical Limits and Practical Implications in Binary Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives exact bounds relating ROC AUC and PR AUC for binary classifiers (given class skew a = N-/N+), showing a high AUC_ROC can coexist with a low AUC_PR on imbalanced data.
Problem
The relationship between ROC AUC and Precision-Recall AUC is unclear, especially on imbalanced datasets, and practitioners lack exact bounds on one metric given the other.
Approach
An exact ROC-to-PR point transformation (x,y) -> (y, y/(y+ax)) is used to derive closed-form min and max of AUC_PR at fixed AUC_ROC (via step-function optimization) and the inverse bounds for AUC_ROC at fixed AUC_PR, plus a rule for comparing two models' rankings on both metrics without computing both AUCs.
Key results
- For a=100 and fixed AUC_ROC=0.875, AUC_PR can range from 0.038 to 0.876 (max/min closed forms, Theorem 3.5).
- For a=1 and AUC_ROC=0.875, AUC_PR ranges from 0.725 to 0.935.
- For a=100 and AUC_PR=0.7, AUC_ROC ranges from 0.697 to 0.999, while for AUC_ROC=0.7 AUC_PR ranges from 0.016 to 0.703.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compare models using AUCPR rather than relying on AUCROC alone | Binary classifiers on highly imbalanced datasets · Binary classification, a=100 | AUCPR: 0.038 to 0.876 | Mixed |
| increase AUCROC from 0.813 to 0.875 | Classifier1 versus Classifier2 · Binary classification, a=100 | AUCPR: 0.514 versus 0.038 | Hurt |
| use AUCROC as the sole performance metric | Binary classifiers on highly imbalanced datasets · Binary classification, a=100 | AUCPR: 0.038 to 0.876 | Mixed |
Surprising findings
- A classifier with AUC_ROC 0.875 can have AUC_PR as low as 0.038 on a=100 data, matching the poor classifier in Davis & Goadrich (2006).
- The proven global minimum of AUC_PR from Boyd et al. (2012) is a special case of the new bounds at c=1.
Limitations
- Results are analytic and depend on the assumed class skew a; the paper gives limited empirical validation on real data.
- Assumes fixed N+ and N- and an increasing ROC curve; the comparison rule is only a sufficient condition.
Experimental setup
Metrics: AUC_ROC AUC_PR
Baselines: Davis & Goadrich 2006 classifier example Boyd et al. 2012 global minimum of AUCPR
Builds on: Davis & Goadrich 2006 ROC-PR relationship Boyd et al. 2012 unachievable region and normalized AUCPR
Closest papers by meaning
- A Closer Look at AUROC and AUPRC under Class Imbalance (NeurIPS 2024 · similarity 0.89)
- Does It Pay to Optimize AUC? (AAAI 2023 · similarity 0.89)
- Towards Decision-Friendly AUC: Learning Multi-Classifier with AUCµ (AAAI 2023 · similarity 0.88)
- Positive-unlabeled AUC Maximization under Covariate Shift (ICML 2025 · similarity 0.88)
- AUC Maximization under Positive Distribution Shift (NeurIPS 2024 · similarity 0.87)
- Preserving AUC Fairness in Learning with Noisy Protected Groups (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.