Research map / Continual learning / Online and replay-based
Linear Mode Connectivity in Multitask and Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Permuted MNIST, 20 tasks | Average accuracy: 80.1 ($\pm$0.51) -> 85.3 ($\pm$0.61) | Helped |
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Permuted MNIST, 20 tasks | Average forgetting: 0.09 ($\pm$0.01) -> 0.06 ($\pm$0.01) | Helped |
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Rotated MNIST, 20 tasks | Average accuracy: 70.8 ($\pm$0.78) -> 82.3 ($\pm$0.68) | Helped |
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Rotated MNIST, 20 tasks | Average forgetting: 0.10 ($\pm$0.02) -> 0.08 ($\pm$0.01) | Helped |
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Split CIFAR-100, 20 tasks | Average accuracy: 59.9 ($\pm$1.81) -> 63.3 ( $\pm$2.21) | Helped |
| Use MC-SGD's replay-plus-linear-connectivity objective instead of Stable SGD | Continual learning · Split CIFAR-100, 20 tasks | Average forgetting: 0.08 ($\pm$ 0.03) -> 0.06 ($\pm$ 0.03) | Helped |
| Start multitask and continual training from different initializations (ablation) | Multitask and continual solutions · Split CIFAR-100 | Linear connectivity (interpolation loss) | Hurt |
| Add Gaussian noise to the second task's input images (ablation) | Multitask and continual solutions · Fashion-MNIST after MNIST | Linear connectivity (interpolation loss) | Hurt |
| Corrupt labels in the second task (ablation) | Multitask and continual solutions · Fashion-MNIST after MNIST | Linear connectivity (interpolation loss) | Hurt |
| Remove classes from the first task's training examples used for multitask training (ablation) | Multitask and continual solutions · Two-task MNIST | Linear connectivity (interpolation loss) | Hurt |
| Use the linear-connectivity objective with replay instead of replay-only methods | Continual learning · Permuted MNIST, Rotated MNIST, and Split CIFAR-100 | Average accuracy: MC-SGD outperforms A-GEM and ER-Reservoir on all three benchmarks | Helped |
| Move from a first-task solution toward a later continual-learning minimum rather than a multitask minimum | Sequential training · Rotated MNIST and Split CIFAR-100 | Previous-task validation loss along interpolation | Hurt |
Closest papers by meaning
- Optimal Task Order for Continual Learning of Multiple Tasks (ICML 2025 · similarity 0.90)
- Understanding the Role of Training Regimes in Continual Learning (NeurIPS 2020 · similarity 0.90)
- Understanding Forgetting in Continual Learning with Linear Regression (ICML 2024 · similarity 0.90)
- Efficient Continual Learning with Modular Networks and Task-Driven Priors (ICLR 2021 · similarity 0.90)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.90)
- Continual Learning with Global Alignment (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.