Research map / Architectures and efficiency / Quantization
Task Vector Quantization for Memory-Efficient Model Merging
Youngeun Kim, Seunghwan Lee, Aecheon Jung, Bogon Ryu, Sungeun Hong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| quantize task vectors to 4 bits instead of quantizing fine-tuned checkpoints | Task arithmetic · 8 classification tasks (ViT-B/32) | average accuracy (%): 4.2 -> 69.1 | Improved |
| quantize task vectors to 3 bits | Task arithmetic · 8 classification tasks (ViT-B/32) | average accuracy (%): 69.2 -> 71.2 | Improved |
| reduce task vector quantization to 2 bits | Task arithmetic · 8 classification tasks (ViT-B/32) | average accuracy (%): 69.2 -> 62.1 | Hurt |
| add residual decomposition (RTVQ, 3-bit base, 2-bit offset) over 2-bit TVQ | Task arithmetic · 8 classification tasks (ViT-B/32) | average accuracy (%): 62.1 -> 70.2 | Improved |
| add residual decomposition (RTVQ) over 2-bit TVQ | EMR-Merging · 8 classification tasks (ViT-L/14) | average accuracy (%): 87.6 -> 90.3 | Improved |
| add residual decomposition (RTVQ) over 2-bit TVQ | EMR-Merging · NYUv2 normal estimation | mean angular error: 45.2 -> 35.0 | Improved |
| add residual decomposition (RTVQ) over 2-bit TVQ | EMR-Merging · NYUv2 segmentation | mIoU: 21.3 -> 34.1 | Improved |
| apply RTVQ instead of FP32 to EMR-Merging | EMR-Merging · 8 classification tasks (ViT-B/32) | average accuracy (%): 88.3 -> 83.2 | Hurt |
| quantize task vectors to 2 bits | ViT-B/32 merging · cross-task accuracy (8 tasks) | accuracy (%): 38.7 -> 48.4 | Improved |
| use RTVQ instead of FP32 task vectors | ViT-B/32 merging · cross-task accuracy (8 tasks) | accuracy (%): 38.7 -> 58.2 | Improved |
| add quantization error correction to RTVQ (ablation) | RTVQ · quantization error | quantization error | Improved |
| allocate more bits to base or offset vector (ablation) | RTVQ with Task arithmetic · 8 classification tasks (ViT-B/32) | average accuracy (%): 69.0 -> 70.9 | No clear effect |
Closest papers by meaning
- Localizing Task Information for Improved Model Merging and Compression (ICML 2024 · similarity 0.88)
- Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation (ICML 2025 · similarity 0.88)
- Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration (NeurIPS 2025 · similarity 0.88)
- CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging (ICML 2025 · similarity 0.88)
- No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces (ICML 2025 · similarity 0.87)
- Less is More: Efficient Model Merging with Binary Task Switch (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
