Research map / Architectures and efficiency / Pruning and efficient tuning
Tangent Model Composition for Ensembling and Continual Fine-tuning
Tian Yu Liu, Stefano Soatto
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compose tangent models by a scalar weight sum (TMC) instead of softmax-ensembling non-linear models | Class-IL continual fine-tuning · C-MIT-67 (Class-IL, 20 tasks) | accuracy: 35.62 -> 48.98 | Improved |
| use softmax ensembling of tangent models (TME) instead of logit composition (TMC) (ablation) | TMC · C-MIT-67 (Class-IL, 20 tasks) | accuracy: 48.98 -> 54.93 | Improved |
| train component models with Rescaled Square Loss instead of CE or MSE (ablation) | TMC · C-Caltech-256/C-MIT-67/C-OxfordPets (10 tasks) | accuracy | Improved |
| increase RSL beta in class-incremental setting (ablation) | TMC · C-Caltech-256/C-MIT-67/C-OxfordPets (10 tasks) | accuracy | Improved |
| increase RSL beta in data-incremental setting (ablation) | TMC · D-Caltech-256/D-MIT-67/D-OxfordPets (10 tasks) | accuracy | Hurt |
| initialize each component model from previous model (TMC-Seq) instead of pre-trained model, data-incremental (ablation) | TMC · D-MIT-67 (Data-IL, 4 tasks) | accuracy: 71.82 -> 72.10 | Improved |
| sequential initialization from previous model, class-incremental (ablation) | TMC · Class-IL (Fig. seqcomp) | accuracy | Hurt |
| linearize only the classification head (TMC-FC) instead of the full network (ablation) | TMC · C-MIT-67 (Class-IL, 20 tasks) | accuracy: 48.98 -> 40.05 | Hurt |
| linearize only the classification head (TMC-FC), pre-trained features sufficient (ablation) | TMC · C-Caltech-256 (Class-IL, 10 tasks) | accuracy: 82.37 -> 83.43 | Improved |
| use tangent fine-tuning instead of non-linear fine-tuning when domains are far from pre-training | TMC · DomainNet (6 domains, avg) | accuracy: 57.13 -> 46.75 | Hurt |
| use TMC without replay buffer vs best replay-based method (TWF) in class-incremental | TMC · C-MIT-67 (Class-IL, 20 tasks) | accuracy: 56.29 -> 48.98 | Hurt |
Closest papers by meaning
- Tangent Transformers for Composition,Privacy and Removal (ICLR 2024 · similarity 0.79)
- Variational Task Vector Composition (NeurIPS 2025 · similarity 0.78)
- Collage: Light-Weight Low-Precision Strategy for LLM Training (ICML 2024 · similarity 0.78)
- Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods (ICML 2025 · similarity 0.78)
- Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation (CVPR 2024 · similarity 0.77)
- Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods (NeurIPS 2025 · similarity 0.77)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
