AI Paper Atlas

Research map / Label-efficient and robust learning / Domain adaptation and generalization

Scalarization for Multi-Task and Multi-Domain Learning at Scale

Amelie Royer, Tijmen Blankevoort, Babak Ehteshami Bejnordi

NeurIPS 2023 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Mixture and distillation for domain shift · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Tune scalarization weights with PBT instead of using uniform weightsDomainNet joint training with ResNet26 at 0.25 width · DomainNet, 6 domainsaverage accuracy: 46.78 ± 0.10 -> 48.01 ± 0.08Improved
Tune scalarization weights with PBT instead of using uniform weightsDomainNet joint training with ResNet26 at original width · DomainNet, 6 domainsaverage accuracy: 48.83 ± 0.08 -> 49.27 ± 0.12Improved
Tune scalarization weights with PBT instead of using uniform weightsDomainNet joint training with ResNet50 at 0.25 width · DomainNet, 6 domainsaverage accuracy: 49.69 ± 0.05 -> 50.69 ± 0.10Improved
Tune scalarization weights with PBT instead of using uniform weightsDomainNet joint training with ResNet50 at original width · DomainNet, 6 domainsaverage accuracy: 51.53 ± 0.06 -> 51.83 ± 0.06Improved
Use PB2-tuned scalarization weights rather than uniform weightsCelebA 8-task MTL with ViT-S/4, 6 layers, full width · CelebAaverage accuracy: 91.23 -> 91.28No clear effect
Use PB2-tuned scalarization weights rather than uniform weightsCelebA 8-task MTL with ViT-S/4, 9 layers, full width · CelebAaverage accuracy: 91.17 -> 91.25No clear effect
Increase model capacity for joint multi-task or multi-domain trainingScalarized MTL/MDL · CIFAR/STL, DomainNet, CelebA, and Taskonomy comparisons with single-dataset modelstest performance relative to single-dataset trainingImproved
Tune task/domain scalarization weights rather than defaulting to uniform weightsScalarized MTL/MDL · High-imbalance task/domain combinationstest performanceImproved
Use a smaller model in the same architecture family to search scalarization weights before scaling upScalarized MTL/MDL · Taskonomy and DomainNetrelative performance ranking across weightsNo clear effect
Correct conflicting gradients during every training iterationUniform scalarization MTL/MDL · CelebA and DomainNet gradient-conflict analysisfinal model performanceNo clear effect
Use uniform weights instead of PBT-tuned weightsDomainNet joint training with ResNet26 at 0.25 width · DomainNet, infograph domainaccuracy: 19.45 ± 0.08 -> 20.46 ± 0.15Improved
Use PBT-tuned weights instead of uniform weightsDomainNet joint training with ResNet26 at 0.25 width · DomainNet, real domainaccuracy: 55.64 ± 0.01 -> 60.43 ± 0.05Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy