Research map / Architectures and efficiency / Pruning and efficient tuning
GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
Yixuan Tang, Yi Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GAPrune (Fisher plus gradient alignment) instead of Domain Fisher pruning (ablation) | Qwen3-Embedding-4B, 50% sparsity · FinMTEB one-shot | average score: 0.4887 -> 0.5224 | Improved |
| use GAPrune instead of General Fisher pruning (ablation) | Qwen3-Embedding-4B, 50% sparsity · FinMTEB one-shot | average score: 0.3623 -> 0.5224 | Improved |
| use GAPrune instead of magnitude pruning (ablation) | Qwen3-Embedding-4B, 50% sparsity · ChemTEB one-shot | average score: 0.7299 -> 0.7462 | Improved |
| use GAPrune instead of magnitude pruning (ablation) | Qwen3-Embedding-4B, 30% sparsity · FinMTEB one-shot | average score: 0.5415 -> 0.5425 | No clear effect |
| use GAPrune instead of magnitude pruning (ablation) | E5-Mistral-7B-Instruct, 50% sparsity · ChemTEB one-shot | average score: 0.6189 -> 0.6206 | No clear effect |
| retrain pruned model for 100 steps after GAPrune | Qwen3-Embedding-4B, 50% sparsity · FinMTEB | average score: 0.5224 -> 0.5594 | Improved |
| retrain pruned model for 100 steps after GAPrune | Qwen3-Embedding-4B, 50% sparsity · ChemTEB | average score: 0.7462 -> 0.7770 | Improved |
| use GAPrune instead of L3 Prune after retraining (ablation) | E5-Mistral-7B-Instruct · ChemTEB prune-and-retrain | average score: 0.6687 -> 0.6773 | Improved |
| use GAPrune instead of Domain Fisher pruning for embedding geometry (ablation) | Qwen3-Embedding-4B · alignment loss, effective dimension | alignment loss: 0.75 -> 0.51 | Improved |
| use GAPrune instead of magnitude pruning for uniformity (ablation) | Qwen3-Embedding-4B · embedding uniformity loss | uniformity loss: -3.30 -> -2.41 | Hurt |
| random pruning of parameters at 50% sparsity (ablation) | Qwen3-Embedding-4B · FinMTEB one-shot | average score: 0.5353 -> 0.2165 | Hurt |
Closest papers by meaning
- Domain Gap Embeddings for Generative Dataset Augmentation (CVPR 2024 · similarity 0.87)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.87)
- POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models (NeurIPS 2026 · similarity 0.86)
- Compact Language Models via Pruning and Knowledge Distillation (NeurIPS 2024 · similarity 0.86)
- Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization (NeurIPS 2025 · similarity 0.86)
- Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
