Research map / Continual learning / Online and replay-based
Computationally Budgeted Continual Learning: What Does Matter?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MSE distillation | Naive replay training · DI-ImageNet2K | average accuracy: 55.9 vs 54.9 | Hurt |
| Add MSE distillation | Naive replay training · CI-ImageNet2K | average accuracy: 64.9 vs 64 | Hurt |
| Add MSE distillation | Naive replay training · CGLM | average stream accuracy: 35.7 vs 26.4 | Hurt |
| Use compute-intensive sampling instead of uniform memory sampling | Budgeted replay training · DI-ImageNet2K, CI-ImageNet2K, and CGLM | accuracy | Hurt |
| Use Class-Balanced instead of Uniform sampling | Budgeted replay training · DI-ImageNet2K | average accuracy: +2.5% | Helped |
| Use FIFO sampling instead of uniform memory sampling | Budgeted replay training · ImageNet2K | ImageNet1K accuracy | Hurt |
| Add distillation | Naive replay training · DI-ImageNet2K, CI-ImageNet2K, and CGLM | accuracy | Hurt |
| Apply FC-layer corrections or calibration | Class-Balanced replay training · DI-ImageNet2K, CI-ImageNet2K, and CGLM | accuracy | Mixed |
| Increase distillation budget from 40 to 400 iterations (ablation) | MSE distillation on CGLM · CGLM | accuracy gap to Naive: 11.41% to 3.85% | Helped |
| Increase the number of time steps to 50 or 200 while normalizing total computation (ablation) | Distillation versus Naive · CGLM, DI-ImageNet2K, and CI-ImageNet2K | accuracy | No clear effect |
| Train only the FC layer rather than the full model (ablation) | ResNet50 with ImageNet1K initialization · CGLM | final accuracy relative to full-model training: 23.73% | Hurt |
| Use Instagram1B+ImageNet1K initialization for FC-only training instead of ImageNet1K initialization (ablation) | FC-only training · CGLM | final accuracy gap to full-model training: 23.73% to 9.45% | Helped |
Closest papers by meaning
- Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling (ICLR 2025 · similarity 0.93)
- Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation (ICLR 2024 · similarity 0.90)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.89)
- Summarizing Stream Data for Memory-Constrained Online Continual Learning (AAAI 2024 · similarity 0.89)
- CLDyB: Towards Dynamic Benchmarking for Continual Learning with Pre-trained Models (ICLR 2025 · similarity 0.89)
- Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.