Research map / Label-efficient and robust learning / Domain adaptation and generalization
Leveraging unlabeled data to predict out-of-distribution performance
Saurabh Garg, Sivaraman Balakrishnan, Zachary Chase Lipton, Behnam Neyshabur, Hanie Sedghi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ATC with a source-validation confidence threshold on unlabeled target data instead of the strongest prior baseline | Fixed classifier accuracy estimation · CIFAR10, CIFAR100, ImageNet, WILDS, MNIST, and BREEDS shifts | MAE: ATC typically achieves the smallest MAE; improves by more than 2× on CIFAR and 3–4× on ImageNet over GDE | Improved |
| Apply source post-hoc Temperature Scaling before estimating accuracy (ablation) | ATC and comparison methods · Multiple distribution shifts | MAE | Improved |
| Use maximum confidence instead of negative entropy as the ATC score (ablation) | ATC · CIFAR100 Synthetic | MAE: 4.94 -> 4.75 | Improved |
| Use negative entropy instead of maximum confidence as the ATC score (ablation) | ATC · ImageNet Natural | MAE: 2.17 -> 0.80 | Improved |
| Fit a robust linear regression to DOC using labeled same-subpopulation target shifts (ablation) | DOC on BREEDS novel subpopulations · Living-17 novel subpopulations | MAE: 24.41 -> 13.26 | Improved |
| Use ATC without regression rather than fine-tuned DOC | Accuracy estimation on BREEDS novel subpopulations · Living-17 novel subpopulations | MAE: 13.26 -> 10.22 | Improved |
| Shift the target invariant-feature class-conditional support while retaining a uniform distribution (ablation) | ATC toy model · Toy model with shifted class-conditional support | accuracy estimation bias | Hurt |
| Introduce novel rather than same subpopulations | All accuracy-estimation methods · BREEDS | MAE | Hurt |
| Pre-train the model before accuracy estimation (ablation) | ATC · CIFAR | MAE | No clear effect |
Closest papers by meaning
- Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.85)
- Sample-Optimal Agnostic Boosting with Unlabeled Data (ICML 2025 · similarity 0.85)
- Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods (NeurIPS 2025 · similarity 0.85)
- quanda: An Interpretability Toolkit for Training Data Attribution Evaluation (NeurIPS 2026 · similarity 0.85)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.85)
- Rethinking Robustness of Model Attributions (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
