Research map / Label-efficient and robust learning / Domain adaptation and generalization
Sparse Deep Transfer Learning for Convolutional Neural Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune source network (sparse main branch) before transfer (ablation) | Standard-TransferNet · MIT Indoor67 | accuracy (%): 69.6% -> 70.1% | Helped |
| prune source network (sparse main branch) before transfer (ablation) | Standard-TransferNet · Flower102 | accuracy (%): 81.7% -> 81.8% | No clear effect |
| add implicit knowledge extra branch (ablation) | Standard-TransferNet · MIT Indoor67 | accuracy (%): 69.6% -> 69.9% | Helped |
| add implicit knowledge extra branch (ablation) | Standard-TransferNet · Flower102 | accuracy (%): 81.7% -> 82.7% | Helped |
| combine sparsity and implicit knowledge (Hybrid-TransferNet) (ablation) | Standard-TransferNet · MIT Indoor67 | accuracy (%): 69.6% -> 71.0% | Helped |
| combine sparsity and implicit knowledge (Hybrid-TransferNet) (ablation) | Standard-TransferNet · Flower102 | accuracy (%): 81.7% -> 83.4% | Helped |
| prune connections in target domain to 95% with total loss | Hybrid-TransferNet · MIT Indoor67 | accuracy (%): 71.0% -> 72.5% | Helped |
| prune connections in target domain to 90% | Hybrid-TransferNet · Flower102 | accuracy (%): 83.4% -> 84.1% | Helped |
| prune connections to 98.5% | Hybrid-TransferNet · Flower102 | accuracy (%): 83.4% -> 82.1% | Hurt |
| prune neurons in FC6 and FC7 as well | Sparse-TargetNet · MIT Indoor67 | accuracy (%): 72.5% -> 69.6% | Hurt |
| keep implicit knowledge during target pruning retraining (ablation) | Sparse-TargetNet · MIT Indoor67 / Flower102 | accuracy (%) | Helped |
| add target-domain pruning to Hybrid-TransferNet | Hybrid-TransferNet · HMDB51 Two-Stream | accuracy (%): 61.4% -> 62.6% | Helped |
Closest papers by meaning
- How Well Do Sparse ImageNet Models Transfer? (CVPR 2022 · similarity 0.92)
- Revisiting Sparse Convolutional Model for Visual Recognition (NeurIPS 2022 · similarity 0.88)
- Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization (ICML 2019 · similarity 0.87)
- PAC-Net: A Model Pruning Approach to Inductive Transfer Learning (ICML 2022 · similarity 0.87)
- Robust Knowledge Transfer via Hybrid Forward on the Teacher-Student Model (AAAI 2021 · similarity 0.87)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.