Research map / Label-efficient and robust learning / Recognition and long-tail
Optimizing Black-box Metrics with Iterative Example Weighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| plug-in example weighting (PI-EW/FW-EG) vs fine-tuning and other baselines | CIFAR-10 label noise · CIFAR-10 noisy labels | test accuracy: 0.781 vs 0.769 (best baseline fine-tuning) | Helped |
| FW-EG with unknown psi vs known psi and baselines | Adult proxy label G-mean · Adult proxy labels | G-mean: 0.685 vs 0.682 (Adaptive Surrogates) | Helped |
| Forward Correction on proxy label noise | Forward Correction · Adult proxy labels | G-mean: 0.214 vs 0.654 cross-entropy train | Hurt |
| FW-EG with unknown psi vs baselines | Adience domain shift F-measure · Adience | F-measure: 0.815 vs 0.781 (fine-tuning) | Helped |
| FW-EG black-box fairness vs Adaptive Surrogates and plug-in | Adult black-box fairness metric · Adult fairness | geometric mean TP/TN by gender: 0.822 vs 0.812 | Helped |
| adding more basis functions including relationship-based features (phi_wf, phi_hs) (ablation) | FW-EG on Adult fairness · Adult fairness | black-box fairness metric: approximately 1% better than fewer basis functions | Helped |
| using PI-EW with better base model quality (log loss) (ablation) | PI-EW on Adult accuracy · Adult accuracy | test accuracy: qualitative trend | Helped |
| FW-EG runtime versus Adaptive Surrogates | FW-EG on Adult proxy labels · Adult proxy labels | runtime: 5x faster | Helped |
| Adaptive Surrogates comparison: Plug-in tuned on validation vs FW-EG on fairness metric | Plug-in [train-val] baseline · Adult fairness | geometric mean TP/TN: 0.812 vs Adaptive Surrogates 0.812 | No clear effect |
Closest papers by meaning
- A General Online Algorithm for Optimizing Complex Performance Metrics (ICML 2024 · similarity 0.87)
- A Separation and Alignment Framework for Black-Box Domain Adaptation (AAAI 2024 · similarity 0.87)
- Principled Algorithms for Optimizing Generalized Metrics in Binary Classification (ICML 2025 · similarity 0.85)
- Label Noise: Ignorance Is Bliss (NeurIPS 2024 · similarity 0.85)
- Gradient-Based Sample Selection for Black-Box Universal Domain Adaptation (AAAI 2025 · similarity 0.84)
- Learning with Noisy Labels Using Hyperspherical Margin Weighting (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.