Research map / Label-efficient and robust learning / Recognition and long-tail
Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a latent score-based reweighting framework that uses diffusion score-similarity differences as a density proxy to upweight low-density samples, improving worst-group accuracy on imbalanced tabular data without group labels.
Problem
ERM-trained classifiers on tabular data underperform on subgroups due to spurious correlations and imbalance; existing mitigation needs group labels or only reweights misclassified samples near the decision boundary (P(Y|X)) rather than balancing the joint distribution P(X,Y).
Approach
Train class-wise score models (EDM-style) on beta-VAE latents of tabular data, and use the difference in aggregated denoising similarity between other-class and same-class score models as a proxy for relative density; weights are a softmax over negative SimDiff, upweighting low-density samples to balance the joint distribution.
Key results
- On five tabular datasets, the method achieves the highest average worst-group accuracy (51.74%) versus ERM (47.56%) and the best baseline, with average mean accuracy 70.80%.
- On ACS Income single-source experiments, average worst-group accuracy of 65.23% vs. 60.79% for ERM.
- Applied to tree-based models (CatBoost, LightGBM, XGBoost) on Adult, worst-group accuracy rises from about 50% to 60-65%, while JTT gives little or negative change.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply latent score-based reweighting instead of χ2-DRO | VAE + MLP · Adult | worst-group accuracy: 50.09 -> 54.79 | Helped |
| apply latent score-based reweighting instead of FAM | VAE + MLP · Bank | worst-group accuracy: 41.09 -> 41.28 | Helped |
| apply latent score-based reweighting instead of FAM | VAE + MLP · Default | worst-group accuracy: 37.12 -> 38.78 | Helped |
| apply latent score-based reweighting instead of KL-DRO | VAE + MLP · Shoppers | worst-group accuracy: 59.74 -> 60.73 | Helped |
| apply latent score-based reweighting instead of χ2-DRO | VAE + MLP · Taxi | worst-group accuracy: 62.43 -> 63.14 | Helped |
| increase selected timesteps from T=5 to T=10 (ablation) | latent score-based reweighting · Adult | worst-group accuracy: 50.82 ±3.17 -> 54.79 ±2.01 | Helped |
| increase selected timesteps from T=10 to T=15 (ablation) | latent score-based reweighting · Adult | worst-group accuracy: 54.79 ±2.01 -> 56.90 ±1.51 | Helped |
| increase selected timesteps from T=15 to T=20 (ablation) | latent score-based reweighting · Adult | worst-group accuracy: 56.90 ±1.51 -> 57.46 ±2.48 | Helped |
| vary selected timestep count from T=5 to T=20 (ablation) | latent score-based reweighting · Adult | mean accuracy: 73.27 ±0.84; 74.33 ±0.28; 74.92 ±0.52; 74.91 ±0.52 | No clear effect |
| vary reweighting temperature τ (ablation) | latent score-based reweighting · datasets in Table 1 | worst-group accuracy | No clear effect |
Surprising findings
- Exact log-likelihood from the probability flow ODE produced extreme values that overemphasized a few high-density points, so the authors switched to a similarity-based proxy.
- JTT gave unstable gains across tree-based models, sometimes decreasing worst-group accuracy.
Limitations
- Relies on hyperparameters T and tau, with sensitivity analysis only on Adult for T.
- Less effective mean accuracy under cross-state generalization on ACS, where causal mechanisms change.
Experimental setup
Datasets: Adult Bank Default Shoppers Taxi US-Wide ACS PUMS Data Phishing
Benchmarks: Adult Bank Default Shoppers Taxi ACS Income (AZ, MA, MI) Phishing Taxi (NYC)
Metrics: mean accuracy worst-group accuracy
Baselines: ERM CVaR-DRO χ2-DRO KL-DRO JTT EIIL FAM SRDO
Builds on: TabSyn EDM JTT score-based generative modeling beta-VAE
Closest papers by meaning
- CLIMB: Class-imbalanced Learning Benchmark on Tabular Data (NeurIPS 2025 · similarity 0.85)
- Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Class (AAAI 2025 · similarity 0.85)
- Learning under Localized Minority Imbalance (NeurIPS 2026 · similarity 0.84)
- Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation (NeurIPS 2024 · similarity 0.83)
- REDUCR: Robust Data Downsampling using Class Priority Reweighting (NeurIPS 2024 · similarity 0.83)
- A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.