Research map / Architectures and efficiency / Pruning and efficient tuning
DiSparse: Disentangled Sparsification for Multitask Model Compression
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Static sparse training · NYU-v2 | mIoU: 23.83 -> 26.48 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Dynamic sparse training · NYU-v2 | mIoU: 24.91 -> 28.16 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Pretrained-model pruning · NYU-v2 | mIoU: 29.19 -> 29.45 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Static sparse training · Cityscapes | mIoU: 38.47 -> 38.21 | Hurt |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Dynamic sparse training · Cityscapes | mIoU: 36.57 -> 38.55 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Pretrained-model pruning · Cityscapes | mIoU: 37.63 -> 40.71 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Static sparse training · Tiny-Taskonomy | mIoU: 26.03 -> 25.53 | Hurt |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Dynamic sparse training · Tiny-Taskonomy | mIoU: 23.87 -> 25.21 | Helped |
| Disentangle task-specific saliency and combine shared-parameter masks with element-wise OR | Pretrained-model pruning · Tiny-Taskonomy | mIoU: 26.71 -> 29.12 | Helped |
| Replace element-wise OR with majority vote as the arbiter (ablation) | DiSparse static · Tiny-Taskonomy | mIoU: 25.53 -> 26.03~(\textbf{0.50}) | Helped |
| Replace element-wise OR with majority vote as the arbiter (ablation) | DiSparse dynamic · Tiny-Taskonomy | mIoU: 25.21 -> 24.97~(-0.24) | Hurt |
Closest papers by meaning
- Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning (ICML 2023 · similarity 0.89)
- GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks (ICML 2018 · similarity 0.88)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.86)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.86)
- DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations (CVPR 2025 · similarity 0.85)
- CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.