Research map / Architectures and efficiency / Pruning and efficient tuning
Efficient Parametrization of Multi-Domain Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace series adapters with parallel adapters | ResNet-26 adapters · Visual Decathlon | mean top-1 accuracy: 77.17 -> 78.07 | Helped |
| apply adapters in early, middle, and late blocks rather than only one block (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 75.32 / 76.40 / 77.40 -> 78.07 | Helped |
| add adapters to the early block (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 75.32 -> 78.07 | Helped |
| add adapters to the middle block (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 76.40 -> 78.07 | Helped |
| add adapters to the late block (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 77.40 -> 78.07 | Helped |
| adapt both convolutional layers in each residual block instead of only the second (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 77.40 -> 78.07 | Helped |
| jointly SVD-compress adapters across domains (ablation) | Parallel adapters on ResNet-26 · Visual Decathlon | mean top-1 accuracy: 78.07 -> 78.36 | Helped |
| use parallel adapters rather than series adapters | pretrained ResNet-26 · Visual Decathlon | Decathlon score: 3159 -> 3412 | Helped |
| append series adapters after pretraining instead of including them during pretraining (ablation) | Series adapters on ResNet · Visual Decathlon, four datasets | mean top-1 accuracy: -1.73 point | Hurt |
| apply dropout with a wider pretrained network (ablation) | Parallel adapters · CIFAR100 | accuracy | Helped |
| increase the number of pretrained ImageNet classes (ablation) | ResNet with parallel adapters · CIFAR100 | accuracy | Helped |
| use adapters instead of full fine-tuning | ResNet transfer learning · CIFAR100 and UCF101 | accuracy | Helped |
Closest papers by meaning
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.87)
- Magnitude Invariant Parametrizations Improve Hypernetwork Learning (ICLR 2024 · similarity 0.85)
- Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks (ICLR 2024 · similarity 0.85)
- On the Parameterization of Second-Order Optimization Effective towards the Infinite Width (ICLR 2024 · similarity 0.85)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.84)
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.