Research map / Architectures and efficiency / Pruning and efficient tuning
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Instant Soup Pruning instead of LTH - Rewind | CLIP ViT-B32 · CIFAR-10, 80% sparsity | accuracy: 87.28 -> 91.20 | Helped |
| Add Instant Soup Pruning instead of Lottery Ticket | BERT Base · QNLI, 70% sparsity | score: 88.9 -> 90.06 ± 1.0 | Helped |
| Add Instant Soup Pruning instead of Lottery Ticket | BERT Base · CoLA, 50% sparsity | score: 53.8 -> 53.61 ± 0.2 | Hurt |
| Add mask denoising to ISP (ablation) | CLIP ViT-B32 · SVHN | accuracy: 96.11±0.21 -> 96.46±0.05 | Helped |
| Add mask denoising to ISP (ablation) | CLIP ViT-B32 · CIFAR-100 | accuracy: 70.32±0.13 -> 71.09±0.17 | Helped |
| Add mask denoising to ISP (ablation) | BERT Base · QQP | score: 89.96±0.39 -> 90.59±0.47 | Helped |
| Add mask denoising to ISP (ablation) | BERT Base · QNLI | score: 89.28±0.87 -> 90.06±1.01 | Helped |
| Increase denoiser count from 0 to 4 (ablation) | ISP with CLIP ViT-B32 · CIFAR-10, 80% sparsity | performance: 70.32 -> 71.15 | Helped |
| Increase denoiser count beyond 4 (ablation) | ISP with CLIP ViT-B32 · CIFAR-10, 80% sparsity | performance: 71.15 -> 71.16 | No clear effect |
| Use IMS instead of the pretrained baseline | CLIP ViT-B32 · Cars | accuracy: 76.43 -> 78.79 | Helped |
| Use IMS instead of the pretrained baseline | BERT Base · MNLI | score: 82.39 -> 83.63 | Helped |
Closest papers by meaning
- Advancing Model Pruning via Bi-level Optimization (NeurIPS 2022 · similarity 0.91)
- Can We Find Strong Lottery Tickets in Generative Models? (AAAI 2023 · similarity 0.91)
- Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost (AAAI 2023 · similarity 0.90)
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.90)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.89)
- Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.