Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Model Fusion via Retrofitting
Phoomraphee Luenam, Andreas Spanopoulos, Amit Sant, Sotiris Anagnostidis, Thomas Hofmann, Sidak Pal Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use KF-Gradient instead of Transformer OTFusion | ViT · CIFAR-100 sharded 4-way, zero-shot | test accuracy: 1.2 -> 41.4 | Improved |
| use HF-Gradient instead of KD | ViT · CIFAR-100 sharded 2-way, zero-shot | test accuracy: 41.0 -> 54.5 | Improved |
| use KF-Gradient instead of KD | ViT · CIFAR-100 sharded 4-way | test accuracy: 47.0 -> 41.4 | Hurt |
| use KF-Linear instead of ensemble | VGG · BloodMNIST sharded 2-way | test accuracy: 88.4 -> 91.0 | Improved |
| use KF-Gradient (25k samples) instead of ZipIt! (25k) | ResNet-50 · ImageNet-1k two subsets, joint | test accuracy: 37.7 -> 58.9 | Improved |
| increase fusion samples for KF-Gradient from 1k to 25k (ablation) | ResNet-50 · ImageNet-1k joint | test accuracy: 25.1 -> 58.9 | Improved |
| add data augmentation to 1k fusion samples (ablation) | KF-Gradient VGG11 · CIFAR-10 non-IID, zero-shot | test accuracy: 73.2 -> 74.5 (uniform) | Improved |
| use Conductance or DeepLIFT importance instead of uniform with 1k samples (ablation) | KF-Gradient VGG11 · CIFAR-10 non-IID, zero-shot | test accuracy: 73.2 -> 71.7 / 71.4 | Hurt |
| use Conductance or DeepLIFT importance instead of uniform with 1k samples plus augmentation (ablation) | KF-Gradient VGG11 · CIFAR-10 non-IID, zero-shot | test accuracy: 74.5 -> 75.3 / 75.2 | Improved |
| use KF-Gradient instead of Transformer OTFusion | ViT · CIFAR-100 full data 2-way, zero-shot | test accuracy: 4.3 -> 63.0 | Improved |
| use KF-Gradient instead of Transformer OTFusion after fine-tuning | ViT · CIFAR-100 full data 2-way | test accuracy: 74.0 -> 75.4 | Improved |
| use KF-Gradient instead of Transformer OTFusion after fine-tuning | ViT · Tiny-ImageNet 2-way | test accuracy: 53.8 -> 54.2 | Improved |
Closest papers by meaning
- Towards Meta-Pruning via Optimal Transport (ICLR 2024 · similarity 0.88)
- Transformer Fusion with Optimal Transport (ICLR 2024 · similarity 0.87)
- Rethinking Layer-wise Model Merging through Chain of Merges (NeurIPS 2026 · similarity 0.85)
- Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts (NeurIPS 2025 · similarity 0.85)
- ModHiFi: Identifying High Fidelity predictive components for Model Modification (NeurIPS 2025 · similarity 0.85)
- $C^2M^3$: Cycle-Consistent Multi-Model Merging (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
