Research map / Continual learning / Online and replay-based
Lifelong Scalable Generative System via Online Maximum Mean Discrepancy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DEMU, a task-free continual learning memory system for DDPM image generators that adds memory units via an MMD-based expansion criterion, prunes overlapping units via graph-based compaction, and optionally expands the diffusion model as a mixture of experts.
Problem
Diffusion models degrade with catastrophic forgetting when trained online on non-stationary data streams without task or class labels (task-free unsupervised continual learning for image generation).
Approach
A Dynamic Expansion Memory Unit that creates new fixed-size memory buffers when the MMD distance between the current buffer and existing buffers exceeds a threshold, selects samples by autoencoder reconstruction error (no labels or pretrained networks), compacts memory by removing the most similar units in a similarity graph, and expands the DDPM into a mixture of DDPM experts (DEMU-D).
Key results
- DEMU-D reaches an average FID of 47.26 across the four class-incremental splits, versus 55.21 for the best baseline CVA (DEMU static: 54.07).
- On CelebA-3DChair DEMU-D achieves FID 79.79 against 132.12 for CGKD-GAN; on high-resolution data DEMU attains FID 92.35 on CelebA-HQ, 81.62 on CACD and 90.52 on FFHQ.
- Ablation on Split MNIST shows FID rises from 20.18 (full DEMU) to 33.79 without memory reduction, indicating memory compaction is needed to avoid memory filling up.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dynamic model expansion | DEMU · Split MNIST, Split Fashion, Split SVHN, and Split CIFAR10 | FID: 20.18 -> 19.23; 43.34 -> 37.64; 62.12 -> 53.49; 83.67 -> 78.70 | Helped |
| Add dynamic model expansion | DEMU · CelebA-3DChair and CelebA-CACD | FID: 81.53 -> 79.79; 73.11 -> 69.85 | Helped |
| Add memory reduction process (ablation) | DEMU-D · Split MNIST | FID: 33.79 -> 19.23 | Helped |
| Add sample selection (ablation) | DEMU-D · Split MNIST | FID: 25.41 -> 19.23 | Helped |
| Add model dynamic expansion (ablation) | DEMU-D · Split MNIST | FID: 23.12 -> 19.23 | Helped |
| Increase the memory expansion threshold λ (ablation) | DEMU · Split MNIST | FID | No clear effect |
| Use smaller memory expansion threshold λ (ablation) | DEMU · Split MNIST | FID | No clear effect |
| Vary memory unit capacity (ablation) | DEMU · Split MNIST | FID | No clear effect |
Surprising findings
- Generative replay baselines LTS and LGM perform worse than static R-DDPM, which uses random sample selection, in the task-free setting.
- Reducing the expansion threshold lambda adds more memory units, but the performance changes only modestly, and fewer than ten memory units already give good results.
Limitations
- Expansion and memory thresholds (lambda, lambda2) are hand-tuned hyperparameters.
- Experiments are restricted to image datasets with FID as the only metric, and memory is capped at 2,000 samples.
Experimental setup
Datasets: MNIST Fashion-MNIST SVHN CIFAR-10 CelebA CACD 3DChair MINI-ImageNet CelebA-HQ FFHQ
Benchmarks: Split MNIST Split Fashion Split SVHN Split CIFAR10 CelebA-CACD CelebA-3DChair Split MINI-ImageNet CelebA-HQ CACD FFHQ
Metrics: FID
Baselines: CVA CGKD-GAN CGKD-WAE LTS LGM R-VAE R-DDPM
Builds on: DDPM Continual Variational Autoencoder CGKD Lifelong Teacher-Student Lifelong Generative Modelling
Closest papers by meaning
- Dynamic Expansion Diffusion Learning for Lifelong Generative Modelling (AAAI 2025 · similarity 0.90)
- Continual Unsupervised Generative Modelling via Online Optimal Transport (AAAI 2025 · similarity 0.88)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.87)
- Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models (CVPR 2024 · similarity 0.87)
- DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture (CVPR 2025 · similarity 0.86)
- Neural Residual Diffusion Models for Deep Scalable Vision Generation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.