Research map / Continual learning / Online and replay-based
Recall-Oriented Continual Learning with Generative Adversarial Meta-Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a recall-oriented continual learning framework in which a lightweight Bayesian neural network acquires each task and a conditional GAN meta-model (GAMM) incrementally learns to generate task-specific parameter chunks via generative replay of parameters, enabling recall of task models at inference.
Problem
The stability-plasticity dilemma in continual learning: methods that preserve old knowledge lose plasticity, and replay of exemplars or pseudo-samples is memory-hungry and fails to capture complex data distributions.
Approach
Separates working memory (a lightweight task-specific BNN trained without replay) from long-term memory (GAMM, a conditional GAN trained over parameter chunks sampled from the BNNs), arguing via volume and separability metrics that learned parameters are a more compact representation than raw data or features; GAMM generates task-specific models on demand at inference.
Key results
- On Split CIFAR-10 task-aware, GAMM reaches 96.73 ACC and -0.13 BWT versus 95.30 ACC for PR-BBB, using 0.9M memory versus 20.2M.
- On Split CIFAR-10 task-agnostic, GAMM achieves 78.49 ACC versus 58.94 for PR-BBB.
- On 5-Datasets, GAMM reaches 93.68 ACC versus 91.32 for HAT and 91.22 for GPM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GAMM instead of BI-R | Continual learning · Split CIFAR-10, task-aware | ACC: 94.99 ± 0.13 -> 96.73 ± 0.08 | Helped |
| Use GAMM instead of PR-BBB | Continual learning · Split CIFAR-10, task-agnostic | ACC: 58.94 ± 2.20 -> 78.49 ± 0.82 | Helped |
| Use GAMM instead of HNET | Continual learning · Split CIFAR-100, task-aware | ACC: 86.50 ± 0.53 -> 87.19 ± 0.27 | Helped |
| Use GAMM instead of PR-BBB | Continual learning · Split CIFAR-100, task-agnostic | ACC: 41.66 ± 0.38 -> 44.75 ± 1.46 | Helped |
| Use GAMM instead of EWC | Continual learning · PMNIST | ACC: 92.01 ± 0.56$^\dagger$ -> 96.40 ± 0.02 | Helped |
| Use GAMM instead of GPM | Continual learning · Split CIFAR-100 | ACC: 72.45 ± 0.54 -> 74.90 ± 0.94 | Helped |
| Use GAMM instead of HAT | Continual learning · 5-Datasets | ACC: 91.32 ± 0.18$^\dagger$ -> 93.68 ± 0.20 | Helped |
Surprising findings
- Learned parameter chunks showed lower complexity (higher separability) than feature vectors despite higher dimensionality.
- Replaying input-level exemplars or pseudo-samples yielded the most negative BWT, indicating raw-data replay is ineffective for preservation.
Limitations
- Task-agnostic performance remains far below task-aware performance on Split CIFAR-100 (44.75 ACC).
- Relies on a lightweight task-specific BNN and per-task compression rates tuned per benchmark (Table of hyperparameters).
- Only evaluated on image classification benchmarks with small architectures.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 Split CIFAR-10 Split CIFAR-100 Permuted MNIST 5-Datasets FMNIST notMNIST
Benchmarks: Split CIFAR-10 Split CIFAR-100 Permuted MNIST 5-Datasets
Metrics: ACC BWT LA memory usage
Baselines: A-GEM ER DGR BI-R HNET PR-BBB EWC HAT GPM
Builds on: SWAG WGAN-GP MeRGAN-RA hypernetwork APD
Closest papers by meaning
- Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective (ICML 2025 · similarity 0.88)
- Learning to Continually Learn with the Bayesian Principle (ICML 2024 · similarity 0.88)
- Task-Free Continual Generation and Representation Learning via Dynamic Expansionable Memory Cluster (AAAI 2024 · similarity 0.88)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.88)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.88)
- Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.