Research map / Architectures and efficiency / Pruning and efficient tuning
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a trainable side network to a frozen base (side-tuning vs features only) (ablation) | Side-tuning · iTaskonomy | average rank: 2.55 -> 1.36 | Improved |
| add the frozen base network to the side network (ablation) | Side-tuning · iTaskonomy | average rank: 2.10 -> 1.36 | Improved |
| use alpha-blending sum instead of element-wise product to merge (ablation) | Side-tuning · iTaskonomy | average rank: 3.64 -> 2.27 | Improved |
| use FiLM merge instead of alpha-blending (ablation) | Side-tuning · iTaskonomy | average rank: 2.27 -> 1.91 | No clear effect |
| use MLP merge instead of alpha-blending (ablation) | Side-tuning · iTaskonomy | average rank: 2.27 -> 2.18 | No clear effect |
| initialize side network from the base instead of Xavier (ablation) | Side-tuning · Taskonomy tasks | performance: p=0.07 | No clear effect |
| initialize side network from the base on harder tasks (excluding texture tasks) (ablation) | Side-tuning · Taskonomy tasks without 2D keypoint, edge, autoencoding | performance: p=0.01 | Improved |
| use side-tuning instead of fine-tuning with 4.9k trajectories | Habitat IL with denoising features · Habitat navigation IL (4.9k trajectories) | navigation reward: 7.5 -> 9.3 | Improved |
| add gradient clipping and RAdam to fine-tuning in RL (ablation) | Fine-tuning · Habitat navigation RL | navigation reward | No clear effect |
| use side-tuning instead of fine-tuning from a Curvature base with 100 images (object classification) | Taskonomy object classification · Taskonomy object classification (100 images) | score: 24.6 -> 24.8 | Improved |
| use side-tuning instead of scratch training with 100 images (normal estimation) | Taskonomy normal estimation · Taskonomy normal estimation (100 images) | loss: 0.323 -> 0.199 | Improved |
| use side-tuning instead of features on SQuAD v2 | BERT · SQuAD v2 (column 1) | match: 49.4 -> 79.6 | Improved |
Later papers that compared with this method
1 later accepted paper reports Side-Tuning as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone (NeurIPS 2023 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.86)
- Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves (CVPR 2025 · similarity 0.85)
- CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning (NeurIPS 2024 · similarity 0.85)
- MIND: Multi-Task Incremental Network Distillation (AAAI 2024 · similarity 0.84)
- TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.84)
- Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
