Research map / Continual learning / Class-incremental
Autoencoder-Based Hybrid Replay for Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AHR, a hybrid replay method that stores compressed exemplars in the latent space of a hybrid autoencoder (HAE) whose latent centroids are placed via charged-particle repulsion, for class-incremental learning.
Problem
Exemplar replay in class-incremental learning requires O(t) memory, while generative replay produces lower-quality pseudo-data; the paper seeks scalable replay that keeps exemplar-level performance.
Approach
Store encoded exemplars in an autoencoder's latent space and decode them for replay, while the autoencoder's latent class centroids are placed incrementally using charged-particle energy minimization and a repulsive force algorithm, so classification is done by nearest centroid in latent space.
Key results
- AHR reaches 97.53% on MNIST(5/2), 93.02% on Balanced SVHN, 77.12% on CIFAR-10, 54.43% on CIFAR-100(10/10) and 48.09% on miniImageNet(20/5), outperforming REMIND+ (52.36% on CIFAR-100) and other listed baselines under equal memory.
- Ablation: AHR-lossless (more exemplars, perfect quality) reaches 56.71% on CIFAR-100 versus 54.43% for AHR, while AHR-lossy-mini (same exemplar count as baselines) reaches 50.29%, close to exemplar-replay baselines.
- Claimed memory reduction to O(0.1t) in the worst case, based on roughly 10x compression in latent space.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AHR instead of REMIND+ | CIL · MNIST (5/2) | accuracy: 95.62 -> 97.53 | Helped |
| Use AHR instead of REMIND+ | CIL · Balanced SVHN (5/2) | accuracy: 92.15 -> 93.02 | Helped |
| Use AHR instead of REMIND+ | CIL · CIFAR-10 (5/2) | accuracy: 75.49 -> 77.12 | Helped |
| Use AHR instead of REMIND+ | CIL · CIFAR-100 (10/10) | accuracy: 52.36 -> 54.43 | Helped |
| Use AHR instead of REMIND+ | CIL · miniImageNet (20/5) | accuracy: 45.02 -> 48.09 | Helped |
| Reduce the exemplar count to baseline size and use lossy compression (ablation) | AHR · CIFAR-100 (10/10) | accuracy: 54.43 -> 50.29 | Hurt |
| Reduce the exemplar count to baseline size and use lossless compression (ablation) | AHR · CIFAR-100 (10/10) | accuracy: 54.43 -> 50.85 | Hurt |
| Use perfect-quality exemplars at AHR's exemplar count (ablation) | AHR · CIFAR-100 (10/10) | accuracy: 54.43 -> 56.71 | Helped |
| Add RFA instead of contrastive latent-space structuring (ablation) | AHR · CIFAR-100 (10/10) | accuracy: 51.98 -> 54.43 | Helped |
| Add RFA instead of Gaussian Mixture Model latent-space structuring (ablation) | AHR · CIFAR-100 (10/10) | accuracy: 49.48 -> 54.43 | Helped |
Surprising findings
- Storing decoded latent exemplars gives a larger gain from the extra exemplar count than from perfect-quality raw exemplars at equal memory, which the authors interpret as evidence that more exemplars matter more than exemplar fidelity.
Limitations
- Evaluated only in the offline task-based CIL setting; task-free and online settings are not evaluated.
- Results are reported only as final accuracy with no variance over multiple seeds beyond SEMs in the table, and the benchmarks are small-scale image datasets.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 CIFAR-100 miniImageNet
Benchmarks: MNIST(5/2) Balanced SVHN(5/2) CIFAR-10(5/2) CIFAR-100(10/10) miniImageNet(20/5)
Metrics: final test accuracy
Baselines: FT FT-E Joint SLDA Gen-C PEC DGR MeRGAN BI-R-SI GD GDumb iCaRL
Builds on: iCaRL EEIL REMIND LwF BiC
Closest papers by meaning
- Tensor Decomposition Based Memory-Efficient Incremental Learning (ICML 2025 · similarity 0.89)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.88)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.87)
- Defying Imbalanced Forgetting in Class Incremental Learning (AAAI 2024 · similarity 0.87)
- Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.