Research map / Label-efficient and robust learning / Recognition and long-tail
CLIMB: Class-imbalanced Learning Benchmark on Tabular Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CLIMB, a benchmark of 73 real-world imbalanced tabular datasets and a unified scikit-learn-style library implementing 29 class-imbalanced learning algorithms, with a 5-fold stratified protocol and Optuna tuning.
Problem
Class-imbalanced learning on tabular data lacks a comprehensive, fair benchmark spanning many datasets, imbalance levels, and algorithm families (resampling, cleaning, cost-sensitive, ensembles) with both accuracy and efficiency evaluation.
Approach
A curated, non-trivial set of 73 naturally imbalanced real-world tabular datasets from OpenML, combined with a unified, tested open-source library of 29 CIL methods evaluated under a standardized 5-fold stratified protocol with per-method hyperparameter search over decision-tree base learners.
Key results
- Undersample ensembles (e.g., SPE, BC, BRF, UBA) match or outperform costlier methods at lower runtime, while many standalone rebalancing methods degrade AUPRC and F1 versus no balancing.
- Adding 10% label noise or 30% missing features to the base model causes a performance drop comparable to increasing the imbalance ratio by 500%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply standalone class rebalancing (undersampling, oversampling, or cost-sensitive reweighting) instead of no balancing | Decision-tree CIL methods · Imbalanced tabular datasets, especially high imbalance | Classification performance | Hurt |
| Use standalone undersampling instead of no balancing | Decision-tree CIL methods · Low-imbalance datasets | AUPRC and macro F1 | Hurt |
| Use cleaning methods rather than aggressive rebalancing | Decision-tree CIL methods · Imbalanced tabular datasets | Classification performance stability | Helped |
| Combine rebalancing with ensemble strategies rather than using standalone rebalancing | CIL methods · Different imbalance levels | Classification performance | Helped |
| Use undersample ensembles rather than standalone undersampling | Decision-tree CIL methods · Imbalanced tabular datasets | Classification performance | Helped |
| Use RUS-based approaches | RUS and its ensembles, including BRF and UBA · Extreme imbalance (IR>50) | Balanced accuracy | Helped |
| Use RUS-based approaches | RUS and its ensembles, including BRF and UBA · Extreme imbalance (IR>50) | AUPRC and macro F1 | Hurt |
| Add 10% label noise | Base model (no balancing) · Imbalanced tabular datasets | Performance: 10\% label noise; increasing the imbalance ratio by 500\% | Hurt |
| Add 30% missing features | Base model (no balancing) · Imbalanced tabular datasets | Performance: 30\% missing features; increasing the imbalance ratio by 500\% | Hurt |
| Further increase class imbalance | IHT · Imbalanced tabular datasets with additional imbalance | Classification performance | Helped |
| Further reduce minority-class size under extreme imbalance | OBA and SMBA · Extreme imbalance | Classification performance | Hurt |
| Use bagging-based rather than boosting-based oversample ensembles | Oversample ensembles · Especially highly imbalanced cases | Classification performance | Helped |
Surprising findings
- Adding 10% label noise or 30% missing features hurts performance about as much as a 500% increase in imbalance ratio.
- Many standalone rebalancing methods (under-, over-sampling, cost-sensitive) underperform no balancing, especially at high imbalance.
Limitations
- Only decision trees used as base learner in main experiments; deep learning and non-tree models not studied.
- Datasets restricted to i.i.d., complete, non-deterministic tabular data, excluding time series and missing-value datasets.
- Robustness analysis is limited to label noise, mean-imputed missing values, and added imbalance.
Experimental setup
Datasets: OpenML tabular datasets (73, naturally imbalanced)
Benchmarks: CLIMB (73 datasets)
Metrics: AUPRC macro F1 balanced accuracy average rank runtime
Baselines: Random Undersampling SMOTE Self-paced Ensemble Balance Cascade Balanced Random Forest EasyEnsemble RUSBoost UnderBagging
Builds on: scikit-learn imbalanced-learn SMOTE Self-paced Ensemble
Closest papers by meaning
- Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Class (AAAI 2025 · similarity 0.87)
- Learning under Localized Minority Imbalance (NeurIPS 2026 · similarity 0.87)
- EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models (NeurIPS 2024 · similarity 0.87)
- TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025 · similarity 0.85)
- Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data (ICML 2025 · similarity 0.85)
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.