Research map / Continual learning / Online and replay-based
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Vary feature similarity between tasks | Two-layer teacher-student continual learning, sigmoidal activation · Synthetic two-task teacher-student setup | forgetting | Mixed |
| Increase feature similarity between tasks | Two-layer teacher-student continual learning · Synthetic two-task teacher-student setup | short-term transfer | Helped |
| Increase feature similarity between tasks | Two-layer teacher-student continual learning · Synthetic two-task teacher-student setup | long-term transfer | Hurt |
| Vary readout similarity while holding feature similarity fixed | Mean-field teacher-student continual learning · Synthetic two-task setup | initial forgetting rate | Mixed |
| Increase the summed feature and readout similarity | Mean-field teacher-student continual learning · Synthetic two-task setup | initial transfer rate | Helped |
| Use equal feature and readout similarity at a fixed summed similarity | Mean-field teacher-student continual learning · Synthetic two-task setup | maximum forgetting | Hurt |
| Use complete task overlap rather than orthogonal features and fully aligned readouts | Mean-field teacher-student continual learning · Synthetic two-task setup | long-time forgetting | Helped |
| Vary readout similarity with identical task features | Mean-field teacher-student continual learning · Synthetic two-task setup | transfer | Mixed |
| Replace the scaled error activation with ReLU (ablation) | Two-layer teacher-student continual learning · Synthetic two-task setup | forgetting versus feature similarity | Mixed |
| Interpolate tasks to intermediate similarity | Two-layer ReLU network · Fashion-MNIST | forgetting | Hurt |
| Switch tasks before convergence on the first task (ablation) | Two-layer teacher-student continual learning · Synthetic two-task setup | initial forgetting rate | Mixed |
| Use orthogonal rather than correlated task features | Mean-field teacher-student continual learning · Synthetic two-task setup | long-term transfer | Helped |
Closest papers by meaning
- Disentangling and mitigating the impact of task similarity for continual learning (NeurIPS 2024 · similarity 0.94)
- The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR 2024 · similarity 0.91)
- Optimal Task Order for Continual Learning of Multiple Tasks (ICML 2025 · similarity 0.90)
- The Importance of Being Lazy: Scaling Limits of Continual Learning (ICML 2025 · similarity 0.89)
- Optimal Protocols for Continual Learning via Statistical Physics and Control Theory (ICLR 2025 · similarity 0.89)
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.