Research map / Architectures and efficiency / Pruning and efficient tuning
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use anti-random rather than random pruning (ablation) | PAT ensemble · CIFAR-10 and CIFAR-100, 16 epochs | accuracy: 75.51 ± 0.06 -> 75.75 ± 0.06; 41.68 ± 0.14 -> 41.85 ± 0.13 | Helped |
| Use anti-random pruning and one-cycle tuning instead of other tested pruning/schedule combinations (ablation) | LeNet-L PAT ensemble · CIFAR-10 and CIFAR-100 | ensemble accuracy | Helped |
| Increase child sparsity up to the critical region (ablation) | LeNet PAT ensemble · CIFAR-10 and CIFAR-100 | ensemble accuracy | Helped |
| Use connection pruning instead of neuron pruning at high sparsity (ablation) | LeNet-L PAT ensemble · CIFAR-10 and CIFAR-100 | ensemble accuracy | Helped |
| Use connection pruning instead of neuron pruning at sparsity up to 50% (ablation) | LeNet-L PAT ensemble · CIFAR-10 and CIFAR-100 | ensemble accuracy | No clear effect |
| Increase ensemble size (ablation) | LeNet-M PAT ensemble · CIFAR-10 and CIFAR-100 | ensemble accuracy | Helped |
| Use child-specific bagging on a 90% training-data subset (ablation) | PAT ensemble · CIFAR-10, ResNet-18 | pairwise output correlation | Helped |
| Use child-specific bagging on a 90% training-data subset (ablation) | PAT ensemble · CIFAR-10, ResNet-18 | accuracy | Hurt |
| Use 50% dropout in place of the PAT child construction (ablation) | LeNet-L · CIFAR-10 and CIFAR-100, 16 epochs | accuracy: 70.06 ± 0.32 vs 75.51 ± 0.06; 28.98 ± 0.33 vs 41.68 ± 0.14 | Hurt |
| Prune children instead of training a single independent model | PAT ensemble · CIFAR-10 and CIFAR-100, 16 epochs, LeNet-L | accuracy: 70.29 ± 0.21 -> 75.51 ± 0.06; 34.39 ± 0.18 -> 41.68 ± 0.14 | Helped |
| Use anti-random pruning and one-cycle tuning | PAT ensemble · CIFAR-10, WRN-28-10, 200 epochs | ECE: 0.023 -> 0.005 | Helped |
| Use anti-random pruning and one-cycle tuning | PAT ensemble · CIFAR-100, WRN-28-10, 200 epochs | accuracy: 82.67 vs 82.8 | Mixed |
Closest papers by meaning
- The Diversified Ensemble Neural Network (NeurIPS 2020 · similarity 0.89)
- BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning (ICLR 2020 · similarity 0.89)
- Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles (AAAI 2025 · similarity 0.88)
- Ensemble Pruning for Out-of-distribution Generalization (ICML 2024 · similarity 0.87)
- Theoretical Limitations of Ensembles in the Age of Overparameterization (ICML 2025 · similarity 0.86)
- MEPSI: An MDL-Based Ensemble Pruning Approach with Structural Information (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.