Research map / Continual learning / Online and replay-based
Task-Free Continual Generation and Representation Learning via Dynamic Expansionable Memory Cluster
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Continual VAE (CAA) with a dual temporary/evolved memory of expandable clusters and a two-step optimisation (generator then inference model) for task-free continual image generation and reconstruction.
Problem
Continual generation and unsupervised representation learning on a task-free data stream without explicit task boundaries, where generative models suffer catastrophic forgetting and memory-based methods have fixed capacity.
Approach
A bio-inspired dual memory: a fixed-size temporary buffer whose samples are transferred into new clusters of an evolving memory when the Jensen-Shannon divergence between aggregate posteriors exceeds a threshold, combined with a two-step VAE-GAN training that updates the generator and inference model on separate objectives.
Key results
- On class-incremental generation with Split CIFAR10, CAA reaches FID 119.58 (reconstruction) and 74.97 (generation) versus 120.83 and 173.73 for OCM, with an average generation FID of 55.21 vs 75.07 for OCM-Dynamic-style baselines in Table 1.
- On Split MNIST classification, Dynamic-CAA achieves 95.23% accuracy vs 94.02% for Dynamic-OCM (Table 3).
- On Permuted MNIST, Dynamic-CAA reaches 86.89% accuracy vs 84.56% for OCM-Dynamic (Table 5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CAA instead of OCM-Dynamic | continual generation model · Split MNIST | generation FID: 45.88 -> 21.46 | Helped |
| Use CAA instead of Reservoir | continual generation model · Split Fashion | generation FID: 99.56 -> 67.28 | Helped |
| Use CAA instead of Reservoir | continual generation model · Split CIFAR10 | generation FID: 93.57 -> 74.97 | Helped |
| Use CAA instead of OCM-Dynamic | continual generation model · MNIST | generation FID: 168.50 -> 245.32 | Hurt |
| Use CAA instead of Reservoir | continual generation model · SVHN | generation FID: 155.06 -> 137.43 | Helped |
| Use CAA instead of Reservoir | continual generation model · CIFAR10 | generation FID: 65.05 -> 87.17 | Hurt |
| Use CAA instead of OCM-Dynamic | continual generation model · ImageNet | reconstruction FID: 82.16 -> 84.97 | Hurt |
| Remove TSOS while keeping Lg′ (ablation) | CAA on Split MNIST · Split MNIST | reconstruction FID: 30.57 -> 33.84 | Hurt |
| Remove TSOS while keeping Lg′ (ablation) | CAA on Split MNIST · Split MNIST | generation FID: 21.46 -> 34.04 | Hurt |
| Replace Lg′ with Lg while retaining TSOS (ablation) | CAA on Split MNIST · Split MNIST | reconstruction FID: 30.57 -> 33.62 | Hurt |
| Replace Lg′ with Lg while retaining TSOS (ablation) | CAA on Split MNIST · Split MNIST | generation FID: 21.46 -> 21.78 | Hurt |
| Remove TSOS and replace Lg′ with Lg (ablation) | CAA on Split MNIST · Split MNIST | reconstruction FID: 30.57 -> 45.72 | Hurt |
Limitations
- Expansion threshold lambda must be tuned per dataset (values between 5 and 30 tested), trading off memory cluster count and performance.
- Prior dynamic expansion methods produce many sub-models with slow inference; the proposed method's scalability to long streams is not deeply analysed.
- Evaluation relies on FID for reconstruction, which is an imperfect measure of reconstruction quality.
Experimental setup
Datasets: Split MNIST Split Fashion MNIST Split SVHN Split CIFAR10 MNIST SVHN CIFAR10 CelebA ImageNet Split CIFAR100 Split MImageNet Permuted MNIST
Benchmarks: Split MNIST Split Fashion MNIST Split SVHN Split CIFAR10 CelebA ImageNet Split CIFAR100 Split MImageNet Permuted MNIST
Metrics: FID classification accuracy
Baselines: OCM OCM-Dynamic Reservoir CNDPM GEM iCaRL MIR GSS CoPE CURL ER + GMED ODDL
Builds on: VAEGAN WGAN-GP Reservoir sampling OCM CN-DPM
Closest papers by meaning
- Online Task-Free Continual Learning via Dynamic Expansionable Memory Distribution (CVPR 2025 · similarity 0.89)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.89)
- Dynamic Expansion Diffusion Learning for Lifelong Generative Modelling (AAAI 2025 · similarity 0.88)
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.88)
- Continual Unsupervised Generative Modelling via Online Optimal Transport (AAAI 2025 · similarity 0.88)
- Recall-Oriented Continual Learning with Generative Adversarial Meta-Model (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.