AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Tangent Model Composition for Ensembling and Continual Fine-tuning

Tian Yu Liu, Stefano Soatto

ICCV 2023 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
compose tangent models by a scalar weight sum (TMC) instead of softmax-ensembling non-linear modelsClass-IL continual fine-tuning · C-MIT-67 (Class-IL, 20 tasks)accuracy: 35.62 -> 48.98Improved
use softmax ensembling of tangent models (TME) instead of logit composition (TMC) (ablation)TMC · C-MIT-67 (Class-IL, 20 tasks)accuracy: 48.98 -> 54.93Improved
train component models with Rescaled Square Loss instead of CE or MSE (ablation)TMC · C-Caltech-256/C-MIT-67/C-OxfordPets (10 tasks)accuracyImproved
increase RSL beta in class-incremental setting (ablation)TMC · C-Caltech-256/C-MIT-67/C-OxfordPets (10 tasks)accuracyImproved
increase RSL beta in data-incremental setting (ablation)TMC · D-Caltech-256/D-MIT-67/D-OxfordPets (10 tasks)accuracyHurt
initialize each component model from previous model (TMC-Seq) instead of pre-trained model, data-incremental (ablation)TMC · D-MIT-67 (Data-IL, 4 tasks)accuracy: 71.82 -> 72.10Improved
sequential initialization from previous model, class-incremental (ablation)TMC · Class-IL (Fig. seqcomp)accuracyHurt
linearize only the classification head (TMC-FC) instead of the full network (ablation)TMC · C-MIT-67 (Class-IL, 20 tasks)accuracy: 48.98 -> 40.05Hurt
linearize only the classification head (TMC-FC), pre-trained features sufficient (ablation)TMC · C-Caltech-256 (Class-IL, 10 tasks)accuracy: 82.37 -> 83.43Improved
use tangent fine-tuning instead of non-linear fine-tuning when domains are far from pre-trainingTMC · DomainNet (6 domains, avg)accuracy: 57.13 -> 46.75Hurt
use TMC without replay buffer vs best replay-based method (TWF) in class-incrementalTMC · C-MIT-67 (Class-IL, 20 tasks)accuracy: 56.29 -> 48.98Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy