AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

Robust and Efficient Continual Model Merging via Global Singular Subspace Separation and Restoration

Juncheol Shin, Byeongho Yu, Semin Kim, Eunhyeok Park

NeurIPS 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Low-rank and model-merging adaptation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GS3R, a data-free framework for continual model merging that separates and restores task updates in a global singular subspace.

Problem

Continual model merging, where task-specific models arrive sequentially, suffers from task balance collapse and substantial performance degradation when standard merging approaches are naively extended.

Approach

A data-free, optimization-compatible framework that separates and restores task contributions within a global singular subspace to balance tasks during sequential merging (details beyond the abstract not available).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add vector restoration to continual SVD merging (ablation)GS3R · ViT-B/32, 20 tasksaccuracy: 55.6 -> 61.7Improved
add global singular subspace separation without restoration (ablation)GS3R · ViT-B/32, 20 tasksaccuracy: 55.6 -> 55.6No clear effect
add global separation on top of restoration (ablation)GS3R · ViT-B/32, 20 tasksaccuracy: 61.7 -> 72.7Improved
truncate restoration state to r=256 (ablation)GS3R · ViT-B/32, 20 tasksaccuracy: 72.7 -> 70.9Hurt
add auxiliary optimization (ablation)GS3R · ViT-B/32, 20 tasksaccuracy: 72.7 -> 73.2Improved
truncate to r=64 instead of full rank (ablation)GS3R · GLUE, Flan-T5-Baseaccuracy: 82.6 -> 83.7Improved
use GS3R instead of NUFILTGS3R · ViT-B/32, 20 tasksaccuracy: 71.0 -> 72.7Improved
use GS3R (no optimization) instead of NUFILTGS3R · ViT-B/32, 8 tasksaccuracy: 83.6 -> 82.7Hurt
increase tolerance to 10^-2 (ablation)GS3R · ViT-B/32, 8 tasksaccuracy: 82.7 -> 81.6Hurt
use GS3R with optimization vs NUFILT under original scorersGS3R + Opt. · LLaVA-1.5-7B multimodalaccuracy: 66.6 -> 70.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy