Research map / Label-efficient and robust learning / Domain adaptation and generalization
Reviewing the Forgotten Classes for Domain Adaptation of Black-Box Predictors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies minority class forgetting in knowledge-distillation-based domain adaptation of black-box predictors and proposes RFC, combining a class-selective small-loss training module with a KD warm-up followed by neighborhood clustering.
Problem
Domain adaptation of black-box predictors (DABP) with only unlabeled target data and black-box source predictions suffers from minority class forgetting, where some classes get 0% accuracy after KD-based training.
Approach
Selection training (ST) adaptively picks classes the model is likely to forget (few predictions plus high entropy) and retrains on small-loss clean samples of those classes; neighborhood clustering (AaD) replaces KD after a DINE warm-up, yielding more balanced learning.
Key results
- RFC reaches 89.2% average accuracy on Office-31 versus 88.2% for BETA, the prior best DABP method.
- RFC reaches 72.9% average accuracy on Office-Home, above BETA at 72.1% and DINE at 69.7%.
- Adding selection training to DINE raises its Office-31 training-phase accuracy from 85.5% to 88.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine selection training and neighborhood clustering | DABP adaptation · Office-31 | accuracy: RFC 89.2; AaD 89.9; BETA 88.2 | Mixed |
| combine selection training and neighborhood clustering | DABP adaptation · Office-Home | accuracy: RFC 72.9; AaD 72.7; BETA 72.1 | Helped |
| combine selection training and neighborhood clustering | DABP adaptation · VisDA-17 | accuracy: RFC 85.2; AaD 88.0; BETA 85.1 | Mixed |
| add selection training | DINE · Office-31 | accuracy: 85.5 -> 88.3 | Helped |
| add selection training | DINE · Office-Home | accuracy: 68.4 -> 70.6 | Helped |
| add selection training | BETA · Office-31 | accuracy: 86.9 -> 88.8 | Helped |
| add selection training | BETA · Office-Home | accuracy: 70.3 -> 71.9 | Helped |
| add selection training (ablation) | RFC · Office-31 | accuracy: 89.2 -> 87.7 | Helped |
| add selection training (ablation) | RFC · Office-Home | accuracy: 72.9 -> 71.5 | Helped |
| add selection training (ablation) | RFC · VisDA-17 | accuracy: 85.2 -> 84.6 | Helped |
| add neighborhood clustering (AaD) (ablation) | RFC · Office-Home | accuracy: 72.9 -> 70.6 | Helped |
| add small-loss sample selection (ablation) | RFC · VisDA-17 | accuracy: 85.2 -> 77.7 | Helped |
Surprising findings
- Existing DABP methods (DINE, BETA) completely forget some minority classes, reaching 0% accuracy on certain classes, despite strong average accuracy.
- Neighborhood clustering (AaD) alone, without a semantically structured warm-up, fails to draw true neighbors of minority classes closer, so selection training is needed alongside it.
Limitations
- Gap between DABP and SFDA remains large on VisDA-17, attributed to weaker target feature semantic structure from distillation.
- Performance on Office-31 W→A is slightly lower than BETA (98.1 vs 99.0), and RFC is not best on all tasks.
- Hyper-parameters (tau, r, warm-up epochs) differ between datasets such as VisDA-17, requiring tuning.
Experimental setup
Datasets: Office-31 Office-Home VisDA-17
Benchmarks: Office-31 Office-Home VisDA-17
Metrics: classification accuracy (%)
Baselines: Source Only NRC AaD G-SFDA LNL-OT LNL-KL HD-SHOT SD-SHOT DINE BETA
Builds on: DINE AaD BETA RSL SHOT
Closest papers by meaning
- A Separation and Alignment Framework for Black-Box Domain Adaptation (AAAI 2024 · similarity 0.90)
- ADU: Adaptive Detection of Unknown Categories in Black-Box Domain Adaptation (CVPR 2025 · similarity 0.89)
- Gradient-Based Sample Selection for Black-Box Universal Domain Adaptation (AAAI 2025 · similarity 0.88)
- Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.87)
- Category-Aware Active Domain Adaptation (ICML 2024 · similarity 0.87)
- Black-Box Forgetting (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.