Research map / 3D vision / Depth and stereo
The VOROS: Lifting ROC Curves to 3D to Summarize Unbalanced Classifier Performance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the VOROS, a volume over a 3D ROC surface that generalizes AUROC to ranges of class imbalance and misclassification cost, and demonstrates it on breast cancer, ultrasound, and credit fraud datasets with classical classifiers.
Problem
AUROC ranks binary classifiers poorly when class imbalance or misclassification costs are highly unbalanced or only known as ranges, since it implicitly assumes balanced conditions and fixed pairwise comparison.
Approach
Lifts the ROC curve to a 3D ROC surface parameterized by cost parameter t, so that the volume over this surface (VOROS) measures classifier performance integrated over a probability measure on t, allowing natural handling of cost/imbalance ranges.
Key results
- On Wisconsin Breast Cancer, VOROS over [0,1] ranks Log 99.8%, Forest 99.8%, Naive 98.4%, baseline 80.7%, while the ranking under restricted cost ranges flips between logistic regression and random forest.
- On the credit fraud data with bounded imbalance and cost ratio, M1 (VOROS 90.1%) beats M2 (89.6%) despite M2 having higher AUROC (85.3% vs 79.5%), and the baseline scores 86.9%.
- The paper gives a closed form for the VOROS of upper-convex-hull vertices (Lemma 23), computable in O(n log n) time via Quickhull.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Compare Logistic Regression with Random Forest using VOROS over t in [0,0.25] rather than all t | Wisconsin Breast Cancer classifiers · Wisconsin Breast Cancer, t in [0,0.25] | VOROS: 99.8% vs 99.9% | Mixed |
| Compare Logistic Regression with Random Forest using VOROS over t in [0.75,1] rather than all t | Wisconsin Breast Cancer classifiers · Wisconsin Breast Cancer, t in [0.75,1] | VOROS: 99.6% vs 99.9% | Helped |
| Compare Logistic Regression with Random Forest on BUSI across cost ranges | BUSI classifiers · BUSI | VOROS: 96.5% vs 98.0%; 96.7% vs 96.7%; 98.4% vs 98.0% | Mixed |
| Choose M1 rather than M2 using VOROS over the specified bounded cost range instead of choosing by AUROC | Credit fraud classifiers · Dal Pozzolo Credit Fraud, t in [999/5999,99/399] | VOROS: 90.1% vs 89.6 % | Helped |
Surprising findings
- A classifier with worse AUROC (M1) can have higher VOROS in a bounded cost range, where it is preferred over the higher-AUROC M2.
- In the credit fraud example, the trivial all-positive baseline scores 86.9% VOROS, outperforming both fitted classifiers in the chosen range.
Limitations
- Only handles binary classification; multiclass generalization left to future work.
- Requires domain experts to supply bounds on misclassification costs and class imbalance.
- Evaluated on only three datasets with default classical classifiers.
Experimental setup
Datasets: Wisconsin Breast Cancer BUSI Credit card fraud (Dal Pozzolo et al.)
Benchmarks: Wisconsin Breast Cancer BUSI Credit card fraud
Metrics: VOROS AUROC
Baselines: AUROC baseline classifiers B0/B1 weighted AUROC
Builds on: AUROC ROC convex hull iso-performance lines weighted AUROC
Closest papers by meaning
- Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective (AAAI 2024 · similarity 0.84)
- A Closer Look at AUROC and AUPRC under Class Imbalance (NeurIPS 2024 · similarity 0.84)
- Beyond the ROC Curve: Classification Trees Using Cost-Optimal Curves, with Application to Imbalanced Datasets (ICML 2024 · similarity 0.83)
- Restoring balance: principled under/oversampling of data for optimal classification (ICML 2024 · similarity 0.83)
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.83)
- Learning under Localized Minority Imbalance (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.