AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Auxiliary Learning by Implicit Differentiation

Aviv Navon, Idan Achituve, Haggai Maron, Gal Chechik, Ethan Fetaya

ICLR 2021 · Label-efficient and robust learning › Recognition and long-tail · approach: Contrastive representation theory · 3 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Learn nonlinear combinations of predefined auxiliary losses (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotTop 1 accuracy: 41.71 ± 0.4 -> 47.07 ± 0.1Improved
Use a deep linear network instead of a shallow linear loss combiner (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotTop 1 accuracy: 41.71 ± 0.4 -> 45.84 ± 0.3Improved
Apply a CNN to spatial maps of per-pixel task losses (ablation)AuxiLearn on NYUv2 · NYUv2mIoU: 20.04 ± 0.38 -> 20.54 ± 0.30Improved
Learn auxiliary labels using an auxiliary set to optimize for main-task generalizationAuxiLearn · CIFAR10 5%Accuracy: 58.2 ± 0.3 -> 60.7 ± 1.3Improved
Constrain the loss-combination network to be monotone (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotTop 1 accuracy: 46.3 ± 0.32 -> 47.07 ± 0.10Improved
Constrain the loss-combination network to be monotone (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 10-shotTop 1 accuracy: 58.84 ± 0.04 -> 59.04 ± 0.22Improved
Tune auxiliary parameters on the training set rather than a separate auxiliary set (ablation)AuxiLearn · Illustrative regression with helpful and harmful auxiliariesGeneralizationHurt
Keep the learned auxiliary network fixed during primary-network retraining instead of jointly updating it (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotAccuracy: 46.7% compared to 40%Hurt
Add an auxiliary-label teacher learned with AuxiLearn rather than using MAXLAuxiLearn on Oxford-IIIT Pet · Oxford-IIIT Pet 30-shotAccuracy: 26.3 ± 0.6 -> 37.0 ± 0.6Improved
Increase the depth of the auxiliary network (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotAccuracyImproved
Allocate too many labeled examples from the training set to the auxiliary set (ablation)AuxiLearn on CUB-200-2011 · CUB 200-2011 5-shotAccuracyHurt
Use a nonlinear loss combiner instead of a linear one (ablation)AuxiLearn on NYUv2 · NYUv2mIoU: 20.04 ± 0.38 -> 20.09 ± 0.34No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy