Research map / Continual learning / Class-incremental

Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning

CVPR 2025 · Continual learning › Class-incremental · approach: Exemplar-efficient class-incremental learning · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MoAL, a pre-trained-model-based analytic class-incremental learning method that uses momentum-based adapter weight interpolation, recursive least-squares knowledge memorization, and prototype-based knowledge rumination to balance plasticity and stability.

Problem

Analytic (least-squares) class-incremental learning with a frozen pre-trained model feature extractor adapts poorly to new classes, giving sub-optimal plasticity when data differ from pre-training distribution.

Approach

Continually adapt the PTM adapter via momentum-based weight interpolation so the feature extractor stays adaptive, keep an recursive analytic classification head for memorization, and use prototype correction plus fine-grained feature generation to revisit and reinforce old-class knowledge without replaying data.

Key results

• MoAL reaches 93.84% average incremental accuracy on CIFAR-100 (T=20 avg), 84.26% avg on ImageNet-R and 72.26% on ImageNet-A, outperforming the best baseline across most settings.
• On ImageNet-A and ImageNet-R, MoAL surpasses the best baseline by an average of 2.29% and 2.92% in average incremental accuracy.
• Ablation on ImageNet-R (T=10) shows knowledge rumination and weight interpolation each add gains over plain ACIL (81.21 to 84.45 with both at T=10 setting in the table).

Measured effects reported in the paper

Surprising findings

• Knowledge distillation alone in place of momentum weight interpolation causes a noticeable performance drop relative to plain ACIL on some benchmarks.
• Frozen-feature analytic learning (ACIL) with a PTM reaches final accuracy on CIFAR-100 close to MoAL, suggesting CIFAR-100 is largely covered by pre-trained generalization.

Limitations

• Momentum interpolation breaks the strict recursion of analytic learning, so exact equivalence to joint training is lost.
• Performance is sensitive to the momentum alpha and buffer dimension d_B; very low alpha degrades results on challenging datasets.

Experimental setup

Datasets: CIFAR-100 ImageNet-A ImageNet-R OmniBenchmark

Benchmarks: CIFAR-100 ImageNet-A ImageNet-R OmniBenchmark

Metrics: average incremental accuracy last-task accuracy

Baselines: L2P DualPrompt ACIL CODA-Prompt LAE DS-AL SimpleCIL Aper EASE LwF

Builds on: ACIL Aper EASE PILOT

Closest papers by meaning

• MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.90)
• PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning (AAAI 2025 · similarity 0.89)
• Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.89)
• Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learning (NeurIPS 2026 · similarity 0.88)
• CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning (CVPR 2025 · similarity 0.88)
• F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning (NeurIPS 2024 · similarity 0.87)