Research map / Optimization / Optimizers
Module-Aware Optimization for Auxiliary Learning
Hong Chen, Xin Wang, Yue Liu, Yuwei Zhou, Chaoyu Guan, Wenwu Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| module-level auxiliary importance instead of single loss weights (Block-L vs Model-L) (ablation) | MAOAL · Pet | accuracy: 67.18 -> 69.61 | Improved |
| module-level importance instead of whole-model weighting (Block-L vs Model-L) (ablation) | MAOAL · CUB-semi | accuracy: 51.16 -> 52.36 | Improved |
| finer partition: each parameter as a module instead of ResNet blocks (ablation) | MAOAL · CUB-semi | accuracy: 52.36 -> 52.48 | No clear effect |
| finer partition: each parameter as a module instead of ResNet blocks (ablation) | MAOAL · Pet | accuracy: 69.61 -> 66.14 | Hurt |
| add module-aware auxiliary rotation loss versus single-loss training | MAOAL vs SLL · Pet-semi (20% labels) | accuracy: 31.81 -> 51.42 | Improved |
| add MAOAL auxiliary learning versus single-loss training | MAOAL vs SLL · CUB-semi (20% labels) | accuracy: 48.15 -> 52.36 | Improved |
| add MAOAL with harmful auxiliary losses versus single-loss training | MAOAL vs SLL · CUB | accuracy: 77.29 -> 77.30 | No clear effect |
| equal-weight auxiliary losses with harmful auxiliaries versus single-loss training | Equal weights · CUB | accuracy: 77.29 -> 71.68 | Hurt |
| MAOAL versus single-loss training with L2 regularizer as auxiliary on sparse data | MAOAL vs SLL · Amazon Beauty | RMSE: 1.1008 -> 1.0873 | Improved |
| MAOAL versus single-loss training with L2 auxiliary on dense data | MAOAL vs SLL · MovieLens-1M | RMSE: 0.9068 -> 0.9045 | No clear effect |
| rotation prediction auxiliary loss on low-resolution CIFAR images | MAOAL vs SLL · CIFAR100 | accuracy: 46.52 -> 47.57 | Improved |
| more auxiliary tasks (NYUv2 depth and normal) with MAOAL versus single-loss | MAOAL vs SLL · NYUv2 segmentation | m-IOU: 32.61 -> 33.88 | Improved |
Closest papers by meaning
- Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost (ICLR 2024 · similarity 0.84)
- Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models (NeurIPS 2023 · similarity 0.83)
- Modular Networks: Learning to Decompose Neural Computation (NeurIPS 2018 · similarity 0.83)
- BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning (NeurIPS 2024 · similarity 0.82)
- Scalable Optimization in the Modular Norm (NeurIPS 2024 · similarity 0.82)
- Careful with that Scalpel: Improving Gradient Surgery with an EMA (ICML 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
