AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation

Aecheon Jung, Seunghwan Lee, Dongyoon Han, Sungeun Hong

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
jointly train merging coefficients with the task-specific layer (vs layer only) (ablation)SyMerge · 20 vision tasks, ViT-B/32average accuracy: 75.0 -> 88.6Improved
jointly train task-specific layer with coefficients (vs coefficients only) (ablation)SyMerge · 8 vision tasks, ViT-B/32average accuracy: 84.6 -> 90.1Improved
adapt multiple layers at once instead of a single layer (ablation)SyMerge · 8 vision tasks, ViT-B/32accuracyHurt
train a single internal transformer layer instead of the classifier (ablation)SyMerge · 8 vision tasks, ViT-B/32accuracyNo clear effect
use SyMerge-learned classifiers with a Task Arithmetic merged encoderTask Arithmetic · 8 vision tasks, ViT-B/32merged accuracy: 69.1 -> 79.6 (+10.5)Improved
use SyMerge-learned classifiers with an AdaMerging encoderAdaMerging · 8 vision tasks, ViT-B/32merged accuracy: 80.1 -> 87.7 (+7.6)Improved
keep SyMerge merged encoder but swap in zero-shot classifier (ablation)SyMerge · 8 vision tasks, ViT-B/32average accuracy: 69.1 -> 82.4Improved
self-labeling with expert predictions instead of entropy minimization (ablation)SyMerge · 8 vision tasks, ViT-B/32Spearman correlation with ground-truth lossImproved
sequential per-task update strategy (ablation)AdaMerging · 8 vision tasks, ViT-B/32average accuracy: 80.1 -> 81.6Improved
use SyMerge for merging models from different initializationsSyMerge · EuroSAT+DTD disjoint basinsaverage accuracy: 6.9 -> 79.1Improved
supervise with a pretrained/individual interpolation lambda 0.8 instead of individual models (ablation)SyMerge · 8 vision tasks, ViT-B/32average accuracy: 90.1 -> about 90.2Improved
jointly train coefficients with classifier (vs classifier only) for init robustness (ablation)SyMerge · 8 vision tasks, ViT-B/32accuracy vs initial coefficientImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy