AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks

Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik

ECCV 2020 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 216 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add a trainable side network to a frozen base (side-tuning vs features only) (ablation)Side-tuning · iTaskonomyaverage rank: 2.55 -> 1.36Improved
add the frozen base network to the side network (ablation)Side-tuning · iTaskonomyaverage rank: 2.10 -> 1.36Improved
use alpha-blending sum instead of element-wise product to merge (ablation)Side-tuning · iTaskonomyaverage rank: 3.64 -> 2.27Improved
use FiLM merge instead of alpha-blending (ablation)Side-tuning · iTaskonomyaverage rank: 2.27 -> 1.91No clear effect
use MLP merge instead of alpha-blending (ablation)Side-tuning · iTaskonomyaverage rank: 2.27 -> 2.18No clear effect
initialize side network from the base instead of Xavier (ablation)Side-tuning · Taskonomy tasksperformance: p=0.07No clear effect
initialize side network from the base on harder tasks (excluding texture tasks) (ablation)Side-tuning · Taskonomy tasks without 2D keypoint, edge, autoencodingperformance: p=0.01Improved
use side-tuning instead of fine-tuning with 4.9k trajectoriesHabitat IL with denoising features · Habitat navigation IL (4.9k trajectories)navigation reward: 7.5 -> 9.3Improved
add gradient clipping and RAdam to fine-tuning in RL (ablation)Fine-tuning · Habitat navigation RLnavigation rewardNo clear effect
use side-tuning instead of fine-tuning from a Curvature base with 100 images (object classification)Taskonomy object classification · Taskonomy object classification (100 images)score: 24.6 -> 24.8Improved
use side-tuning instead of scratch training with 100 images (normal estimation)Taskonomy normal estimation · Taskonomy normal estimation (100 images)loss: 0.323 -> 0.199Improved
use side-tuning instead of features on SQuAD v2BERT · SQuAD v2 (column 1)match: 49.4 -> 79.6Improved

Later papers that compared with this method

1 later accepted paper reports Side-Tuning as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy