Research map / Continual learning / Online and replay-based
Self-Supervised Models Are Continual Learners
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CaSSLe distillation to SimCLR | SimCLR · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 48.9 -> 58.3 | Helped |
| Add CaSSLe distillation to Barlow Twins | Barlow Twins · ImageNet100, class-incremental, 5 tasks | linear top-1 accuracy: 63.1 -> 68.2 | Helped |
| Add CaSSLe distillation to BYOL | BYOL · ImageNet100, data-incremental, 5 tasks | linear top-1 accuracy: 74.0 -> 73.3 | Hurt |
| Add CaSSLe distillation to Barlow Twins | Barlow Twins · DomainNet, domain-incremental, 6 tasks | linear top-1 accuracy: 50.3 -> 55.5 | Helped |
| Add swapped-view distillation (ablation) | CaSSLe with SimCLR · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 58.3 -> 49.3 | Hurt |
| Remove predictor head (ablation) | CaSSLe with SimCLR · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 58.3 -> 52.6 | Hurt |
| Add swapped-view distillation (ablation) | CaSSLe with Barlow Twins · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 60.4 -> 57.4 | Hurt |
| Remove predictor head (ablation) | CaSSLe with Barlow Twins · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 60.4 -> 57.3 | Hurt |
| Add swapped-view distillation (ablation) | CaSSLe with BYOL · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 62.2 -> 52.0 | Hurt |
| Remove predictor head (ablation) | CaSSLe with BYOL · CIFAR100, class-incremental, 5 tasks | linear top-1 accuracy: 62.2 -> 58.6 | Hurt |
| Add CaSSLe distillation to BYOL | BYOL · ImageNet100, data-incremental, 5 tasks | linear top-1 accuracy: 74.0 -> 73.3 | Mixed |
Closest papers by meaning
- Probing Representation Forgetting in Supervised and Unsupervised Continual Learning (CVPR 2022 · similarity 0.89)
- Representational Continuity for Unsupervised Continual Learning (ICLR 2022 · similarity 0.88)
- Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.88)
- Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation (ICLR 2024 · similarity 0.87)
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.87)
- RanDumb: Random Representations Outperform Online Continually Learned Representations (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.