Research map / Label-efficient and robust learning / Recognition and long-tail
RotoGrad: Gradient Homogenization in Multitask Learning
Adrián Javaloy, Isabel Valera
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine gradient-magnitude scaling with task-specific rotations | RotoGrad on NYUv2 · NYUv2 | mIoU: 38.05 -> 39.32 | Improved |
| remove gradient-magnitude scaling (Rotate Only) (ablation) | RotoGrad on NYUv2 · NYUv2 | mIoU: 38.05 -> 39.63 | Improved |
| remove task-specific rotations (Scale Only) (ablation) | RotoGrad on NYUv2 · NYUv2 | mIoU: 38.05 -> 38.89 | Improved |
| use gradient-magnitude scaling without rotations (ablation) | RotoGrad on NYUv2 · NYUv2 | Abs. depth: 0.53 -> 0.54 | Hurt |
| use task-specific rotations without gradient-magnitude scaling (ablation) | RotoGrad on NYUv2 · NYUv2 | Mean relative normal-task improvement: -25.70 -> -6.56 | Mixed |
| rotate a smaller feature subspace (ablation) | RotoGrad on CIFAR10 · CIFAR10 | F1-score | Hurt |
| make the rotation optimizer learn faster relative to the network optimizer (ablation) | RotoGrad on multi-MNIST · multi-MNIST sum task | test error | Hurt |
| reduce the rotation learning rate below the reported sweet spot (ablation) | RotoGrad on multi-MNIST · multi-MNIST sum task | test error: 5e-4 | No clear effect |
Closest papers by meaning
- Gradient Surgery for Multi-Task Learning (NeurIPS 2020 · similarity 0.86)
- GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks (ICML 2018 · similarity 0.86)
- EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization (NeurIPS 2021 · similarity 0.86)
- Selective Task Group Updates for Multi-Task Optimization (ICLR 2025 · similarity 0.85)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.84)
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
