Research map / Architectures and efficiency / RNNs and normalization
Slice-and-Pack: Tailoring Deep Models for Customized Requirements
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Slice-and-Pack, which slices pre-trained classifiers into per-class feature-extractor fragments using limited data and packs user-selected fragments into a compact multi-class model within the learnware market setting.
Problem
In the learnware market, a user's required abilities (classes) are often spread across several existing models, and offering all models costs more and needs more hardware; data for developers is limited due to privacy.
Approach
Slice each model layer by layer into class-specific fragments using ability pooling (l2,1-norm sparsification and unit selection) and ability adaption (linear repair plus adaption loss), then pack selected fragments with an encoder-decoder fusion layer and per-fragment decoders while keeping fragments frozen.
Key results
- On CIFAR tasks with VGG16 and k=20, S&P (T) reaches e.g. 81.40% on T2 and 84.65% on T3 with about 37% and 37% of ensemble parameters, versus finetuning at 68.35% and 67.00%.
- On text tasks with k=10, S&P outperforms the ensemble on T7 (74.02% vs 73.32%) with about half the parameters (35.3e4 vs 72.0e4).
- The equivalent market expansion rate reaches about 200,000x at acceptable error 0.2 and about 600,000x at 0.3 (k-dependent).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | VGG16 · T1 CIFAR10, k=20 | accuracy: 66.99±2.33 -> 76.37±5.49 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | ResNet34 · T1 CIFAR10, k=20 | accuracy: 71.56±4.80 -> 80.43±6.01 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | VGG16 · T2 CIFAR100, k=20 | accuracy: 68.35±8.21 -> 81.40±4.15 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | ResNet34 · T2 CIFAR100, k=20 | accuracy: 73.30±9.60 -> 84.50±7.92 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | VGG16 · T3 CIFAR100, k=20 | accuracy: 67.00±5.31 -> 79.55±4.27 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | ResNet34 · T3 CIFAR100, k=20 | accuracy: 73.55±6.44 -> 84.65±3.92 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | VGG16 · T4 CIFAR100, k=20 | accuracy: 58.72±5.39 -> 73.72±5.38 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | ResNet34 · T4 CIFAR100, k=20 | accuracy: 63.32±5.47 -> 79.46±4.12 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | VGG16 · T5 CIFAR100, k=20 | accuracy: 42.11±3.44 -> 65.03±4.75 | Helped |
| Slice and pack class-specific fragments using threshold-based unit selection rather than fine-tune one original model | ResNet34 · T5 CIFAR100, k=20 | accuracy: 50.81±4.44 -> 71.73±4.25 | Helped |
| Use S&P instead of combining all original models with an ensemble | RNN · T6 TREC, k=10 | accuracy: 83.47 -> 81.56 | Hurt |
| Use S&P instead of combining all original models with an ensemble | CNN · T7 SST-5, k=10 | accuracy: 73.32 -> 74.02 | Helped |
Surprising findings
- On T7 with k=10, S&P beats the ensemble method while using fewer parameters, despite the ensemble accessing all original models.
- Fragments from different classes select significantly different units, yet the packed model still reaches accuracy close to the full ensemble.
Limitations
- Experiments are limited to small classification models (VGG16, ResNet34, CNN, RNN) and four datasets.
- Packed models are about 5 points less accurate than the full ensemble on image tasks.
- Standard deviations are high because variance is computed across different source models rather than random seeds.
Experimental setup
Datasets: CIFAR10 CIFAR100 TREC SST-5
Benchmarks: CIFAR10 CIFAR100 TREC SST-5
Metrics: accuracy parameter count equivalent expansion rate packing time
Baselines: Finetune CA-MKD NetGraft Ensemble
Builds on: learnware paradigm NetGraft CA-MKD
Closest papers by meaning
- Towards Making Learnware Specification and Market Evolvable (AAAI 2024 · similarity 0.82)
- Learnware Specification via Dual Alignment (ICML 2025 · similarity 0.82)
- On the Ability of Developers' Training Data Preservation of Learnware (NeurIPS 2024 · similarity 0.81)
- Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks (NeurIPS 2024 · similarity 0.81)
- CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation (CVPR 2025 · similarity 0.81)
- Stealix: Model Stealing via Prompt Evolution (ICML 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.