Research map / Label-efficient and robust learning / Test-time adaptation
AUC Maximization under Positive Distribution Shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an AUC maximization method for imbalanced binary classification under a positive distribution shift, using labeled positives and unlabeled data from both training and test distributions, without needing the test class-prior.
Problem
AUC maximization degrades when the test distribution differs from training due to a positive-conditional shift (negative-conditional fixed), and no labeled test or negative training data are available.
Approach
Shows that the test AUC under a positive distribution shift can be written using positive training density, marginal training density, and marginal test density, yielding a simple pairwise loss that is optimized with PU-style sampling and does not require the test class-prior.
Key results
- On SVHN with test class-prior 0.3, the proposed method reaches 0.736 test AUC versus 0.519 for PURR and 0.503 for CE.
- Proposed method is best or statistically comparable in 11 of 12 main-table settings.
- On CIFAR10 with test class-prior 0.3, the method attains 0.874 AUC versus 0.710 for PURR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed positive-shift AUC objective instead of the strongest reported baseline | AUC maximization under positive distribution shift · SVHN, πte=0.1 | test AUC: 0.524 -> 0.554 | Helped |
| Use the proposed positive-shift AUC objective instead of the strongest reported baseline | AUC maximization under positive distribution shift · CIFAR10, πte=0.3 | test AUC: 0.750 -> 0.874 | Helped |
| Remove the second term from the proposed loss (ablation) | Ours · MNIST, varying πtr | test AUC | Hurt |
| Remove the second term from the proposed loss (ablation) | Ours · FashionMNIST, varying πtr | test AUC | Hurt |
| Remove the second term from the proposed loss (ablation) | Ours · CIFAR10, varying πtr | test AUC | Hurt |
| Remove the second term from the proposed loss (ablation) | Ours · SVHN, varying πtr | test AUC | No clear effect |
| Estimate rather than use the true training class-prior (ablation) | Ours · MNIST, average over πte | test AUC: 0.826 -> 0.845 | Helped |
| Estimate rather than use the true training class-prior (ablation) | Ours · FashionMNIST, average over πte | test AUC: 0.879 -> 0.857 | Hurt |
| Estimate rather than use the true training class-prior (ablation) | Ours · SVHN, average over πte | test AUC: 0.650 -> 0.652 | Helped |
| Estimate rather than use the true training class-prior (ablation) | Ours · CIFAR10, average over πte | test AUC: 0.809 -> 0.811 | Helped |
| Apply non-negative loss correction (ablation) | Ours · Across four image datasets | test AUC: Ours: 0.827, 0.879, 0.650, 0.809; Ours w/ nn: 0.816, 0.879, 0.650, 0.792 | No clear effect |
| Apply lower-bound loss correction using πte (ablation) | Ours · Across four image datasets | test AUC: 0.827 -> 0.885; 0.879 -> 0.954; 0.650 -> 0.572; 0.809 -> 0.829 | Mixed |
Surprising findings
- The standard non-negative PU correction did not help in this AUC framework, because zero is not a tight lower bound of the loss.
- Increasing the input class-prior above the true training prior often improved performance, attributed to partial overlap of positive types between training and test.
Limitations
- Relies on the positive distribution shift assumption; not guaranteed to work if the negative-conditional density changes.
- Performance on the more complex datasets (SVHN, CIFAR10) is modest in absolute terms, e.g., 0.554 AUC at test prior 0.1 on SVHN.
- Requires the training class-prior to be known or estimated; the non-negative correction is ineffective, and the tighter correction needs the test class-prior.
Experimental setup
Datasets: MNIST FashionMNIST SVHN CIFAR10 HReadmission Hypertension
Benchmarks: MNIST FashionMNIST SVHN CIFAR10 HReadmission Hypertension
Metrics: test AUC F1 score G-mean of TPR and TNR
Baselines: CE nnPU puAUC PULA AnnPU ApuAUC APULA CpuAUC PURR BPURR
Builds on: PU learning (nnPU) AUC maximization with unlabeled data (puAUC) PURR PULA
Closest papers by meaning
- Positive-unlabeled AUC Maximization under Covariate Shift (ICML 2025 · similarity 0.94)
- Interplay of ROC and Precision-Recall AUCs: Theoretical Limits and Practical Implications in Binary Classification (ICML 2024 · similarity 0.87)
- Semi-Supervised Learning for Maximizing the Partial AUC (AAAI 2020 · similarity 0.86)
- Does It Pay to Optimize AUC? (AAAI 2023 · similarity 0.86)
- A Closer Look at AUROC and AUPRC under Class Imbalance (NeurIPS 2024 · similarity 0.83)
- Preserving AUC Fairness in Learning with Noisy Protected Groups (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.