Research map / Continual learning / Online and replay-based
Recasting Continual Learning as Sequence Modeling
Soochan Lee, Jaehyeon Son, Gunhee Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sequence-modeling Transformer MCL instead of the strongest class-incremental baseline | Transformer · CIFAR-100, 20-task 5-shot meta-test | classification error (%): 76.6^{\pm0.3} -> 82.8^{\pm0.8} | Hurt |
| Use sequence-modeling Transformer MCL instead of PN | Transformer · Omniglot, 20-task 5-shot meta-test | classification error (%): 3.8^{\pm0.1} -> 13.7^{\pm0.6} | Hurt |
| Use sequence-modeling Transformer MCL instead of PN | Transformer · CASIA, 20-task 5-shot meta-test | classification error (%): 0.4^{\pm0.0} -> 0.4^{\pm0.0} | No clear effect |
| Use sequence-modeling Transformer MCL instead of GeMCL | Transformer · MS-Celeb-1M, 20-task 5-shot meta-test | classification error (%): 33.3^{\pm0.2} -> 30.0^{\pm0.2} | Improved |
| Use standard Transformer instead of Linear TF | MCL sequence model · CASIA, 100-task 5-shot | classification error (%): 2.3^{\pm0.1} -> 1.0^{\pm0.0} | Improved |
| Use standard Transformer instead of OML | MCL sequence model · Sine, 100-task 5-shot | mean squared error: 0.0498^{\pm0.0004} -> 0.0031^{\pm0.0002} | Improved |
| Use standard Transformer instead of OML | MCL sequence model · Rotation, 100-task 5-shot | 1 - cos(angular error): 0.524^{\pm0.087} -> 0.031^{\pm0.001} | Improved |
| Replace the CNN image encoder with ResNet-18 (ablation) | Transformer · Omniglot, 20-task 5-shot | classification error (%): 13.7^{\pm0.6} -> 25.5^{\pm1.2} | Hurt |
| Use longer combinatorial class codes (C=2 or C=3) instead of C=1 (ablation) | Transformer · CASIA, 20-task 5-shot | classification error (%): 0.4^{\pm0.0} -> 0.5^{\pm0.1}; 0.6^{\pm0.0} | Hurt |
| Increase Transformer model size from 2.4M to 36.0M parameters (ablation) | Transformer · CASIA, 100-task 5-shot | classification error (%): 1.5^{\pm0.0} -> 0.8^{\pm0.0} | Improved |
Closest papers by meaning
- Learning to Continually Learn with the Bayesian Principle (ICML 2024 · similarity 0.90)
- On the Stability-Plasticity Dilemma in Continual Meta-Learning: Theory and Algorithm (NeurIPS 2023 · similarity 0.90)
- Compositional Language Continual Learning (ICLR 2020 · similarity 0.89)
- Look-ahead Meta Learning for Continual Learning (NeurIPS 2020 · similarity 0.89)
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.87)
- Scalable Language Model with Generalized Continual Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
