Research map / Label-efficient and robust learning / Recognition and long-tail
Group-robust classification
Improves worst-group and subclass performance via distribution balancing, priors and invariant augmentation.
251 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Recognition and long-tail, keyword cluster “label · classification · classifier”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
80 papers in 2023–24 and 71 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 6.4% to 5.4%.
Open questions the data points to
- Papers report ERM on Waterbirds (worst-group accuracy) anywhere from 38.9 to 72.6: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Personalized Microblog Sentiment Classification via Multi-Task Learning (AAAI 2016 · 37 citations)
- 2017 · ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases (CVPR 2017 · 3,529 citations)
- 2018 · Who Said What: Modeling Individual Labelers Improves Classification (AAAI 2018 · 200 citations)
- 2019 · Precision-Recall versus Accuracy and the Role of Large Data Sets (AAAI 2019 · 319 citations)
- 2020 · Ensemble Distribution Distillation (ICLR 2020 · 124 citations)
- 2021 · Your "Flamingo" is My "Bird": Fine-Grained, or Not (CVPR 2021 · 132 citations)
- 2022 · Balanced MSE for Imbalanced Visual Regression (CVPR 2022 · 185 citations)
- 2023 · Learning Sample Relationship for Exposure Correction (CVPR 2023 · 42 citations)
- 2024 · A Closer Look at AUROC and AUPRC under Class Imbalance (NeurIPS 2024 · 34 citations)
- 2025 · CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss (CVPR 2025 · 15 citations)
- 2026 · Positive Distribution Shift as a Framework for Understanding Tractable Learning (ICML 2026 · 0 citations)
Earliest papers here
- Personalized Microblog Sentiment Classification via Multi-Task Learning (AAAI 2016 · 37 citations)
- Robust Text Classification in the Presence of Confounding Bias (AAAI 2016 · 32 citations)
- Drosophila Gene Expression Pattern Annotations via Multi-Instance Biological Relevance Learning (AAAI 2016 · 7 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Waterbirds · worst-group accuracy (%) (higher is better)
11 papers, 2021–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 86.7 | Just Train Twice: Improving Group Robustness without Training Group Information (ICML 2021 · 69 citations) |
| 2022 | 88.5 | Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations (ICML 2022 · 14 citations) |
| 2023 | 90.5 | Avoiding spurious correlations via logit correction (ICLR 2023 · 3 citations) |
| 2024 | 92.3 | Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation (CVPR 2024 · 4 citations) |
| 2025 | 92.9 | Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions (ICLR 2025) |
CelebA · worst-group accuracy (%) (higher is better)
9 papers, 2021–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 81.1 | Just Train Twice: Improving Group Robustness without Training Group Information (ICML 2021 · 69 citations) |
| 2022 | 88.8 | Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations (ICML 2022 · 14 citations) |
| 2024 | 89.5 | Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference (ICML 2024 · 1 citation) |
CivilComments · worst-group accuracy (higher is better)
5 papers, 2023–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 70.3 | Avoiding spurious correlations via logit correction (ICLR 2023 · 3 citations) |
| 2024 | 77.6 | The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations (NeurIPS 2024 · 2 citations) |
What has been tried
211 papers here report 2,211 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 8%, hurt 21%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Set all margins to 1 instead of learning separate margins | Adience and KOA test sets · accuracy | Adience: 0.6302 -> 0.6306; KOA: 0.6673 -> 0.6612 | No clear effect | CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss (CVPR 2025 · 15 citations) |
| Replace CE with NAW-CE, without adding regularization | RAF-DB · overall accuracy | 87.42 -> 88.17 | Helped | Navigating Label Ambiguity for Facial Expression Recognition in the Wild (AAAI 2025 · 11 citations) |
| Add contrastive learning | KineticsSounds · MAP | 69.32% -> 71.76% | Helped | Facilitating Multimodal Classification via Dynamically Learning Modality Gap (NeurIPS 2024 · 10 citations) |
| Apply ORD with CTabSyn rather than CTabSyn without ORD | Adult · Avg of 4 accuracy | 79.80 -> 80.67 | Helped | Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Class (AAAI 2025 · 9 citations) |
| replace mean-based confidence thresholding with truncated thresholding | Synthetic, symmetric label bias · test error | 1.51 -> 0.77; 3.60 -> 0.91 | Helped | Mitigating Label Bias in Machine Learning: Fairness through Confident Learning (AAAI 2024 · 9 citations) |
| Use HCA-d instead of HCA-add | IMDB-WIKI-DIR · bMAE (All) | 12.86 -> 12.70 | Helped | Deep Imbalanced Regression via Hierarchical Classification Adjustment (CVPR 2024 · 9 citations) |
| Add the word-caused bias reduction operation | Legal Verdict · Acc | 85.81 -> 86.33 | Helped | De-biased Attention Supervision for Text Classification with Causality (AAAI 2024 · 8 citations) |
| add the duplication mechanism to the screening heads | Criteo · AUC | 0.8451 -> 0.8477 | Helped | Online Conversion Rate Prediction via Multi-Interval Screening and Synthesizing under Delayed Feedback (AAAI 2024 · 5 citations) |
| Add fairness-aware generative augmentation to BYOL's traditional augmentation | CelebA (T=Smiling, S=Male) · accuracy and fairness | Helped | Distributionally Generative Augmentation for Fair Facial Attribute Classification (CVPR 2024 · 5 citations) | |
| use 25 groups instead of 2 groups | AgeDB-DIR · MAE | 7.11 -> 6.87 | Helped | Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression (AAAI 2025 · 4 citations) |
| add contrastive learning | ASW-REAL · weighted average F1 (β = 1) | 0.57 ± 0.08 -> 0.74 ± 0.16 | Helped | T-NET: Weakly Supervised Graph Learning for Combatting Human Trafficking (AAAI 2024 · 4 citations) |
| use correctly classified samples for combination instead of tuning selected-sample proportion q | CelebA · worst group accuracy | 81.9±0.7 -> 76.11±0 | Hurt | Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation (CVPR 2024 · 4 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| ERM | Waterbirds | worst-group accuracy | 38.9 – 72.6 | 5 papers, e.g. NeurIPS 2024 and ICML 2021 |
| JTT | CMNIST | worst-group accuracy (%) | 67.3 – 74.5 | 3 papers, e.g. ICML 2024 and CVPR 2024 |
What reviewers will expect
Datasets and baselines named in the 57 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 8 of 57 papers
- CIFAR-100 · 6 of 57 papers
- CelebA · 5 of 57 papers
- Waterbirds · 4 of 57 papers
- ImageNet · 4 of 57 papers
- MNIST · 3 of 57 papers
- SVHN · 3 of 57 papers
- MultiNLI · 3 of 57 papers
Baselines
- Focal loss · 3 of 57 papers
- ResNet50 · 3 of 57 papers
- ConR · 3 of 57 papers
- RankSim · 3 of 57 papers
- SMOTE · 3 of 57 papers
- ERM · 3 of 57 papers
- Group DRO · 3 of 57 papers
- CE · 2 of 57 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 151 papers; from OpenAlex and the papers' affiliations).
- New York University · 5 papers
- Meta · 5 papers
- Seoul National University · 5 papers
- University of Chinese Academy of Sciences · 4 papers
- Georgia Institute of Technology · 4 papers
- Chinese Academy of Sciences · 4 papers
- Google · 4 papers
- Google DeepMind · 4 papers
- Massachusetts Institute of Technology · 4 papers
- Carnegie Mellon University · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Partial and multi-label learning (345 papers)
- Loss design and classifier theory (273 papers)
- Learning from unlabeled and positive-unlabeled data (240 papers)
- Confidence calibration objectives (129 papers)
- Deep active learning (76 papers)
Elsewhere in the topic
- Multilingual language representation pretraining (507 papers)
- Contrastive representation theory (495 papers)
- Dense features and foundation priors (482 papers)
- Correspondence and causal representation learning (447 papers)
- Data augmentation and generalization bias (409 papers)
- Recommendation representation learning (207 papers)
- Margin-based face recognition losses (124 papers)
- Supervised person re-identification (108 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).