Research map / Architectures and efficiency / Pruning and efficient tuning
SlimSAM: 0.1% Data Makes Segment Anything Slim
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compresses SAM-B's image encoder by 50-77% with alternate embedding/bottleneck pruning and distillation plus disturbed Taylor importance, using only 10k SA-1B images on one GPU.
Problem
Compressing the Segment Anything Model normally requires training a new lightweight encoder from scratch with large data; naive pruning plus fine-tuning degrades sharply at high pruning ratios when data is scarce.
Approach
Alternately prune and distill decoupled sub-structures (embedding dimensions, then bottleneck dimensions) so that intermediate features keep consistent dimensionality for distillation, and estimate label-free importance via Taylor gradients computed on noise-perturbed image embeddings (disturbed Taylor importance) so pruning criteria match the distillation objective.
Key results
- SlimSAM-77 reaches 67.40% MIoU on SA-1B with 9.1M params and 23G MACs, using 10k training images on a single Titan RTX.
- SlimSAM-50 reaches 72.33% MIoU with 26M params and 98G MACs, versus 71.15% for Taylor pruning at the same ratio.
- Compared with EdgeSAM and MobileSAM, SlimSAM uses about 10x less training data; compared with EfficientSAM, about 1100x less.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SlimSAM-50 instead of EdgeSAM | SAM-B compression on SA-1B · SA-1B | MIoU: 65.96% -> 72.33% | Helped |
| Use SlimSAM-77 instead of EdgeSAM | SAM-B compression on SA-1B · SA-1B | MIoU: 65.96% -> 67.40% | Helped |
| Replace Taylor importance with disturbed Taylor importance (ablation) | one-step pruning on SAM-B · SA-1B, 77% pruning | MIoU: 62.04% -> 62.31% | Helped |
| Replace Taylor importance with disturbed Taylor importance (ablation) | SlimSAM-77 · SA-1B, 77% pruning | MIoU: 63.63% -> 64.48% | Helped |
| Add bottleneck feature alignment to final-image-embedding distillation (ablation) | SlimSAM-77 step 1 · SA-1B | MIoU: 65.10% -> 66.32% | Helped |
| Add embedding feature alignment to final-image-embedding distillation (ablation) | SlimSAM-77 step 2 · SA-1B | MIoU: 63.91% -> 64.48% | Helped |
| Use global bottleneck pruning with Gaussian normalization rather than local pruning (ablation) | SlimSAM-77 · SA-1B, 77% pruning | MIoU: 64.38% -> 64.48% | Helped |
| Reduce training data from 10k to 5k images (ablation) | SlimSAM-50 · SA-1B, 50% pruning | MIoU: 72.33% -> 71.89% | Hurt |
| Reduce training data from 10k to 2k images (ablation) | SlimSAM-50 · SA-1B, 50% pruning | MIoU: 72.33% -> 69.79% | Hurt |
| Reduce training data from 10k to 5k images (ablation) | SlimSAM-77 · SA-1B, 77% pruning | MIoU: 67.40% -> 64.47% | Hurt |
| Increase training iterations from 50k to 100k (ablation) | SlimSAM-50 · SA-1B, 50% pruning | MIoU: 70.83% -> 72.33% | Helped |
| Increase training iterations from 100k to 200k (ablation) | SlimSAM-77 · SA-1B, 77% pruning | MIoU: 64.43% -> 67.40% | Helped |
Surprising findings
- Scratch distillation with the same budget collapses to 1.34% MIoU at 77% pruning, showing pruning from the pretrained SAM is essential under limited data.
Limitations
- Performance ceiling is bounded by the original SAM; the method has not surpassed SAM.
- Only 10k training images were used due to hardware limits; more data likely helps, especially at high pruning ratios.
- Gains from the data-efficient design are reported mostly on SA-1B MIoU with single-point prompts.
Experimental setup
Datasets: SA-1B (10k-image subset)
Benchmarks: SA-1B
Metrics: MIoU parameter count MACs inference speedup
Baselines: SAM-H SAM-L SAM-B FastSAM MobileSAM EfficientSAM EdgeSAM Random Pruning Magnitude Pruning Hessian Pruning Taylor Pruning Scratch Distillation
Builds on: Taylor pruning knowledge distillation Segment Anything Model
Closest papers by meaning
- TinySAM: Pushing the Envelope for Efficient Segment Anything Model (AAAI 2025 · similarity 0.87)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.85)
- SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression (ICML 2025 · similarity 0.85)
- Breaking the Compression Ceiling: Data-Free Pipeline for Ultra-Efficient Delta Compression (NeurIPS 2025 · similarity 0.84)
- EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything (CVPR 2024 · similarity 0.84)
- InfoSAM: Fine-Tuning the Segment Anything Model from An Information-Theoretic Perspective (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.