Research map / Architectures and efficiency / Pruning and efficient tuning

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning

ICML 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NegMerge, which merges task vectors from many hyperparameter-varied fine-tunes on the forget set by keeping only sign-consistent elements, then negates the merged vector to unlearn with task arithmetic.

Problem

Task Arithmetic-style unlearning by negating a task vector is highly sensitive to hyperparameters, requiring extensive validation to pick one fine-tuned model, and discards the other candidates.

Approach

Instead of selecting a single fine-tuned model, average the task vectors from all validation-stage fine-tunes while keeping only elements whose signs agree across all of them (sign consensus), and negate this sparse merged vector from the pre-trained model; the mask can be updated incrementally so storing all models is unnecessary.

Key results

• On CLIP ViT-B/32 the method reduces forget-set accuracy to 20.76% versus 23.63% for the single best Task Arithmetic model, while retain accuracy stays around 60%.
• On CIFAR-10 with ResNet-18 (10% random forgetting), NegMerge achieves an average gap of 1.07 to the retrained model, below SalUn (1.15) and Task Arithmetic (1.62).
• The merged task vector modifies only about 5-10% of weight elements, versus 50-60% for other merging methods.

Measured effects reported in the paper

Surprising findings

• Merging more task vectors increases sparsity and, in their experiments, improves unlearning performance.
• Using only sign-conflicting elements (the reverse selection) substantially degrades unlearning, with forget accuracy on Cars rising to 40.2% versus 27.4% for sign-consistent elements.

Limitations

• The method lacks theoretical justification, as acknowledged by the authors.
• The sign-unanimity rule is a simple fixed choice; partial-consensus variants were not explored.
• Linear Task Arithmetic results could not be reported for ViT-L/14 due to computational constraints.

Experimental setup

Datasets: SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD ImageNet CIFAR-10 CUB-200-2011 Tiny ImageNet

Benchmarks: SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD ImageNet CIFAR-10 CUB-200-2011 Tiny ImageNet

Metrics: forget set accuracy retain set accuracy test accuracy MIA-Efficacy Avg. Gap merging time sparsity

Baselines: Task Arithmetic Uniform Merge Greedy Merge TIES-Merging MagMax Linear Task Arithmetic Fine-tuning Random Labeling Gradient Ascent Influence Unlearning l1-sparse Boundary Shrink

Builds on: Task Arithmetic Model Soups TIES-Merging MagMax Linear Task Arithmetic

Closest papers by meaning

• Machine Unlearning under Overparameterization (NeurIPS 2025 · similarity 0.86)
• The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation (NeurIPS 2026 · similarity 0.86)
• DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers (NeurIPS 2025 · similarity 0.86)
• From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization (NeurIPS 2025 · similarity 0.86)
• LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.86)
• When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.85)