Research map / Architectures and efficiency / RNNs and normalization
Multi-Task Zipping via Layer-wise Neuron Sharing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Hessian-based neuron matching and incoming-weight updates instead of random neuron pairing and random weight choice | LeNet-300-100-Dense on MNIST · MNIST | test error: 33.47% error with random sharing; 0.95% increase in test error with MTZ | Helped |
| Fully share the hidden layers and retrain | LeNet models · MNIST | test error: $1.57\%$ / $1.60\%$ -> $1.64\%; $1.80\%$ / $1.81\%$ -> $1.83\%; $0.89\%$ / $0.95\%$ -> $0.93\%; $1.27\%$ / $1.28\%$ -> $ | No clear effect |
| Fully share all hidden layers rather than adaptively share a subset | VGG-16 models for ImageNet and CelebA · ImageNet and CelebA | test error: ImageNet 10.31% -> 14.07%; CelebA 8.50% -> 11.09% | Hurt |
| Adaptively share layers while limiting the error increase to less than 0.5% | VGG-16 models for ImageNet and CelebA · ImageNet and CelebA | test error: increase in test errors of 0.44% on ImageNet and 0.45% on CelebA; 39.61% of parameters shared | Helped |
| Inherit pretrained models through MTZ rather than jointly train a fully shared multi-task VGG-16 | VGG-16 for ImageNet and CelebA · ImageNet and CelebA | test error: 14.88% -> 14.07%; 13.29% -> 11.09% | Helped |
| Merge five task-specific ResNet28 models at the reported 90% compression ratio | Five-task ResNet28 · Five-task ResNet28 mean | test error: 17.95\% -> 18.20\% | Hurt |
| Use layer-wise neuron sharing and incoming-weight updating to fully merge hidden layers | LeNet-300-100-Dense without retraining · MNIST | test error increase: 0.95%; 1.50% | Helped |
Closest papers by meaning
- ZipIt! Merging Models from Different Tasks without Training (ICLR 2024 · similarity 0.85)
- Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.84)
- Task Singular Vectors: Reducing Task Interference in Model Merging (CVPR 2025 · similarity 0.84)
- Rethinking Layer-wise Model Merging through Chain of Merges (NeurIPS 2026 · similarity 0.83)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.83)
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.