Research map / Label-efficient and robust learning / Domain adaptation and generalization
Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STDW, a self-training gradual domain adaptation method that uses a time-varying weight rho (0 to 1) to shift the loss from source to target batches, evaluated on rotated/color-shifted MNIST, Portraits, Cover Type and CIFAR-10-C/100-C.
Problem
Gradual domain adaptation suffers from inefficient knowledge migration across intermediate domains, error accumulation from pseudo-labels, and incomplete intermediate data under large source-target shifts.
Approach
Self-training over a sequence of intermediate domains where each update mixes losses on the current and next domain batches with a weight rho that increases monotonically from 0 to 1, plus cyclic batch matching between neighboring domains and per-batch dynamic pseudo-labeling.
Key results
- STDW reports 97.6% on Rotated MNIST, 98.3% on Color-Shift MNIST, 87.1% on Portraits and 74.2% on Cover Type, the best among listed UDA and GDA baselines.
- On CIFAR-10-C (severity 5) STDW reaches a mean error of 15.5% versus 15.6% for GTTA-MIX and 43.5% source-only.
- On CIFAR-100-C STDW reaches a mean error of 25.8% versus 28.9% for GTTA-MIX and 46.4% source-only.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply STDW instead of the strongest listed baseline | Gradual domain adaptation · Rotated MNIST | accuracy: 87.5 -> 97.6 | Helped |
| apply STDW instead of the strongest listed baseline | Gradual domain adaptation · Color-Shift MNIST | accuracy: 91.8 -> 98.3 | Helped |
| apply STDW instead of the strongest listed baseline | Gradual domain adaptation · Portraits | accuracy: 86.1 -> 87.1 | Helped |
| apply STDW instead of the strongest listed baseline | Gradual domain adaptation · Cover Type | accuracy: 73.5 -> 74.2 | Helped |
| apply STDW instead of GTTA-MIX | CIFAR-10-C adaptation with WideResNet-28 · CIFAR-10-C severity 5 mean | error rate: 15.6 -> 15.5 | Helped |
| apply STDW instead of GTTA-MIX | CIFAR-100-C adaptation with ResNeXt-29 · CIFAR-100-C severity 5 mean | error rate: 28.9 -> 25.8 | Helped |
| use equal-increment monotonic varrho schedule instead of a fixed 0.5 weight (ablation) | STDW · Rotated MNIST, 4 adaptation steps | accuracy: 84.1 -> 88.1 | Helped |
| use equal-increment monotonic varrho schedule instead of independently random weights (ablation) | STDW · Rotated MNIST, 4 adaptation steps | accuracy: 86.1 -> 88.1 | Helped |
| use equal-increment monotonic varrho schedule instead of sorted-random weights (ablation) | STDW · Rotated MNIST, 4 adaptation steps | accuracy: 85.7 -> 88.1 | Helped |
| use equal-increment monotonic varrho schedule instead of a fixed 0.5 weight (ablation) | STDW · Portraits, 4 adaptation steps | accuracy: 84.7 -> 85.3 | Helped |
| use equal-increment monotonic varrho schedule instead of independently random weights (ablation) | STDW · Portraits, 4 adaptation steps | accuracy: 84.1 -> 85.3 | Helped |
| use equal-increment monotonic varrho schedule instead of sorted-random weights (ablation) | STDW · Portraits, 4 adaptation steps | accuracy: 84.5 -> 85.3 | Helped |
Surprising findings
- Reported main-table accuracies for STDW on Rotated MNIST (97.6%) and Color-Shift MNIST (98.3%) exceed the ablation table values for the same setting (e.g., 88.1% at 4 steps), which the paper does not reconcile.
- Ablation shows the Rand and Sorted schedules sometimes match or exceed the monotonic schedule at certain step counts, so the benefit of the monotonic rho schedule is not uniform.
Limitations
- Relies on availability and quality of intermediate domains.
- Introduces scheduling hyperparameters for rho that require tuning.
- Theoretical analysis depends on strong assumptions (smoothness, strong convexity, bounded Wasserstein distances) and experiments cover only image and tabular data.
Experimental setup
Datasets: Rotated MNIST Color-Shift MNIST Portraits Cover Type CIFAR-10-C CIFAR-100-C
Benchmarks: Rotated MNIST Color-Shift MNIST Portraits Cover Type CIFAR-10-C CIFAR-100-C
Metrics: accuracy error rate
Baselines: DANN DeepCoral DeepJDOT GST IDOL GOAT GGF GNF TENT-continual AdaContrast CoTTA GTTA-MIX
Builds on: gradual self-training (GST) IDOL GOAT DeepJDOT
Closest papers by meaning
- Understanding Self-Training for Gradual Domain Adaptation (ICML 2020 · similarity 0.91)
- Weight Diffusion for Future: Learn to Generalize in Non-Stationary Environments (NeurIPS 2024 · similarity 0.88)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.87)
- Enhancing Evolving Domain Generalization through Dynamic Latent Representations (AAAI 2024 · similarity 0.87)
- Dynamic Target Distribution Estimation for Source-Free Open-Set Domain Adaptation (AAAI 2025 · similarity 0.87)
- Diffusion-Driven Progressive Target Manipulation for Source-Free Domain Adaptation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.