Research map / Large language models / Language, safety and interpretability
OPIC: Enhancing Language Model Merging via Optimizing In-Context Capability
Jie He, Weidong Bao, Chao Chen, Zhengyi Zhong, Shuai Zhang, Ji Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inter-layer refinement matrix A (ablation) | OPIC · 4-task merge, Qwen2.5-1.5B | mean normalized score: 77.80% -> 80.70% | Improved |
| add task scaling vector B (ablation) | OPIC · 4-task merge, Qwen2.5-1.5B | mean normalized score: 78.30% -> 80.70% | Improved |
| add both A and B (hierarchical refinement) beyond global scalar C (ablation) | OPIC · 4-task merge, Qwen2.5-1.5B | mean normalized score: 76.40% -> 80.70% | Improved |
| widen refinement search range from [0,1] to [0,1.2] and [0,1.5] (ablation) | OPIC · 4-task merge, Qwen2.5-1.5B | normalized score: max difference 0.36% | No clear effect |
| use OPIC validation-free optimization instead of DOGE | OPIC vs DOGE · 4-task merge, Qwen2.5-1.5B | mean normalized score: 69.62% -> 80.73% | Improved |
| integrate OPIC into TIES | TIES · 4-task merge, Qwen2.5-1.5B | mean normalized score: 77.63% -> 78.80% | No clear effect |
| integrate OPIC into DARE | DARE · 4-task merge, Qwen2.5-1.5B | mean normalized score: 78.32% -> 76.10% | No clear effect |
| integrate OPIC into TIES | TIES · ANLI, Qwen2.5-1.5B | normalized score: ANLI 80.0% -> 100.1%; COQA 58.9% -> 49.0% | Mixed |
| TIES and DARE pruning-based merging versus simple Task Arithmetic | TIES/DARE · 4-task merge, Qwen2.5-1.5B | mean normalized score: 79.56% -> 77.63% (TIES), 78.32% (DARE) | Hurt |
| OPIC versus TIES/DARE on out-of-domain reasoning | OPIC · ARC, Qwen2.5-1.5B | accuracy: 0.761 (TIES) vs 0.653 | Hurt |
| OPIC on MMLU versus Task Arithmetic | OPIC · MMLU, Qwen2.5-1.5B | accuracy: 0.332 -> 0.544 | Improved |
| OPIC evolutionary search versus grid search compute | OPIC · merge cost | GPU hours: ~0.2 -> ~0.54 | Hurt |
Closest papers by meaning
- Model Merging in Pre-training of Large Language Models (NeurIPS 2025 · similarity 0.90)
- MergeBench: A Benchmark for Merging Domain-Specialized LLMs (NeurIPS 2025 · similarity 0.89)
- Activation-Informed Merging of Large Language Models (NeurIPS 2025 · similarity 0.88)
- Channel Merging: Preserving Specialization for Merged Experts (AAAI 2025 · similarity 0.88)
- Model Fusion through Bayesian Optimization in Language Model Fine-Tuning (NeurIPS 2024 · similarity 0.88)
- Mixture-Trained Merging for Unified Multi-Objective Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
