Research map / Continual learning / Continual fine-tuning of LLMs
Null-Space Filtering for Data-free Continual Model Merging: Preserving Transparency, Promoting Fidelity
Zihuan Qiu, Lei Wang, Yang Cao, Runtong ZHANG, Bing Su, Yi Xu, Fanman Meng, Linfeng Xu, Qingbo Wu, Hongliang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add null-space projection to naive task-vector merging (ablation) | naive merging · ViT-B/32, 8 tasks | ACC: 62.1 -> 80.0 | Improved |
| add null-space projection to naive merging (ablation) | naive merging · ViT-B/32, 20 tasks | ACC: 34.3 -> 67.0 | Improved |
| add null-space projection to naive merging (ablation) | naive merging · ViT-B/32, 8 tasks | BWT: -18.5 -> -1.7 | Improved |
| add data-free low-rank objective to null-space filter (ablation) | NUFILT · ViT-B/32, 8 tasks | ACC: 80.0 -> 83.6 | Improved |
| add data-free low-rank objective to null-space filter (ablation) | NUFILT · ViT-B/32, 20 tasks | ACC: 67.0 -> 71.0 | Improved |
| add data-free low-rank objective to null-space filter (ablation) | NUFILT · ViT-B/32, 8 tasks | BWT: -1.7 -> -2.7 | Hurt |
| add null-space filter to data-free objective (ablation) | NUFILT · ViT-B/32, 8 tasks | ACC: 75.8 -> 83.6 | Improved |
| add null-space filter to data-free objective (ablation) | NUFILT · ViT-B/32, 20 tasks | BWT: -20.6 -> -8.9 | Improved |
| add data-free objective alone to naive merging (ablation) | naive merging · ViT-B/32, 8 tasks | ACC: 62.1 -> 75.8 | Improved |
| increase null-space rank r_p excessively (ablation) | NUFILT · ViT-B/32 | ACC | Hurt |
| increase low-rank adaptation rank r_l (ablation) | NUFILT · ViT-B/32 | ACC | Improved |
| use NUFILT instead of OPCM | OPCM · ViT-L/14, 20 tasks | ACC: 76.0 -> 84.7 | Improved |
Closest papers by meaning
- Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging (NeurIPS 2025 · similarity 0.89)
- No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces (ICML 2025 · similarity 0.88)
- Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors (ICML 2025 · similarity 0.87)
- Contribution-Aware Structured Sparsity for Model Merging (NeurIPS 2026 · similarity 0.87)
- MINGLE: Mixture of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging (NeurIPS 2025 · similarity 0.87)
- C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
