Research map / Continual learning / Online and replay-based

Re-evaluating Continual Learning with Few-Shot Adaptation

NeurIPS 2026 · Continual learning › Online and replay-based · approach: Unified and unsupervised continual frameworks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes few-shot evaluation of continual learning stability and plasticity, a per-shot plasticity metric (SAUCE), and foresight meta-learning that meta-trains on a short look-ahead of future tasks; experiments on small visual CL sequences with ResNet-18/MLP.

Problem

Standard continual learning evaluation measures stability as 0-shot forgetting and plasticity as 0-shot accuracy on the current task, which misses retrieval of seemingly forgotten knowledge and the rate at which models adapt to new tasks.

Approach

Evaluate every checkpoint on all tasks after k-shot adaptation (k=0,1,2,5,10), define SAUCE to measure the relative rate of few-shot adaptation, and meta-initialize the learner on unseen future-task examples (foresight meta-learning) rather than replayed past data.

Key results

• SGD recovers much of its forgetting under 5-shot adaptation on backward tasks, nearly matching replay methods in every setting while 0-shot accuracy is near random.
• Standard plasticity (0-shot current-task accuracy) is roughly identical across CL methods, whereas 10-shot forward transfer differentiates them; EWC loses plasticity on long sequences.
• Adding foresight meta-learning yields a steady decrease in average SAUCE over the sequence on every sequence except Rot-MNIST-20, indicating learning-to-learn behavior (reported qualitatively).

Surprising findings

• SGD, which has no stability mechanism, recovers most prior-task accuracy with 5 shots, suggesting forgetting is largely a retrieval issue.
• Replay-based methods (ER, DER++) appear to reduce per-shot plasticity relative to SGD and regularization methods despite strong few-shot performance ceilings.

Limitations

• Evaluation at scale limits the exhaustiveness of hyperparameter search, especially for meta-learning settings.
• Mechanism behind learning-to-learn improvement is not explored.
• Only visual classification studied; in-context adaptation for foundation models left to future work.

Experimental setup

Datasets: MNIST CIFAR100 Seq-MNIST-5 Seq-CIFAR100-10 Seq-CIFAR100-20 Struct-CIFAR100-20 Rot-MNIST-20

Benchmarks: Seq-MNIST-5 Seq-CIFAR100-10 Seq-CIFAR100-20 Struct-CIFAR100-20 Rot-MNIST-20

Metrics: 0-shot accuracy k-shot accuracy (k=0,1,2,5,10) backward/forward transfer SAUCE (per-shot plasticity) AUAC

Baselines: SGD ER DER++ EWC A-GEM MER meta-SGD

Builds on: MAML Reptile Mammoth Experience Replay DER++ La-MAML

Closest papers by meaning

• Stabilizing Zero-Shot Prediction: A Novel Antidote to Forgetting in Continual Vision-Language Tasks (NeurIPS 2024 · similarity 0.90)
• STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.88)
• Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.88)
• PaLoRA: Paced Low-Rank Adaptation for Continual Learning (NeurIPS 2026 · similarity 0.88)
• Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off (AAAI 2025 · similarity 0.88)
• BECAME: Bayesian Continual Learning with Adaptive Model Merging (ICML 2025 · similarity 0.88)