Research map / Continual learning / Class-incremental

MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning

AAAI 2025 · Continual learning › Class-incremental · approach: Exemplar-efficient class-incremental learning · 15 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MOS, which trains task-specific adapters merged via EMA across tasks and uses a training-free self-refined adapter retrieval at inference to resist parameter- and retrieval-level forgetting in pre-trained-model-based class-incremental learning.

Problem

Class-incremental learning with pre-trained models suffers catastrophic forgetting at both the parameter level (drift from iterative updates of lightweight modules) and the retrieval level (mistakenly selecting irrelevant prompts/adapters at inference).

Approach

Train one adapter per task on a frozen ViT, merge each new adapter with previous ones via an exponential-moving-average rule during training, and at test time iteratively re-select the adapter using the model's own predicted task ID until it is self-consistent; the final prediction ensembles the first-task adapter with the selected adapter.

Key results

• On the table benchmarks with ViT-B/16-IN21K, MOS reaches 89.25 last-stage accuracy on CIFAR100 B0 Inc5 versus 88.55 for SLCA and 85.80 for EASE.
• MOS reports 80.05 last-stage accuracy on OmniBenchmark B0 Inc30, versus 74.85 for EASE and 74.10 for SLCA.
• MOS trails EASE on VTAB (92.79 vs 93.55 last-stage), which the paper leaves unexplained beyond the table.

Measured effects reported in the paper

Surprising findings

• Simple prototype-based SimpleCIL is a strong baseline, outperforming several prompt-based methods such as L2P and DualPrompt on CUB and ImageNet-R settings.

Limitations

• Retrieval relies on the first-task adapter's predictions, so errors in the initial selector can propagate.
• Gains are smaller or absent on some benchmarks (e.g., VTAB where EASE is slightly better).
• Stores per-class Gaussian statistics for classifier alignment, which is an additional memory cost not fully discussed in the main text.

Experimental setup

Datasets: CIFAR100 CUB200 ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB

Benchmarks: CIFAR100 CUB200 ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB

Metrics: average incremental accuracy last-stage accuracy

Baselines: Finetune Finetune Adapter LwF L2P DualPrompt CODA-Prompt SimpleCIL APER SLCA EASE FOSTER MEMO

Builds on: L2P DualPrompt CODA-Prompt APER SimpleCIL

Closest papers by meaning

• Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.90)
• Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning (CVPR 2025 · similarity 0.89)
• Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learning (NeurIPS 2026 · similarity 0.89)
• CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning (CVPR 2025 · similarity 0.88)
• Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning (CVPR 2025 · similarity 0.88)
• Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)