Research map / Continual learning / Online and replay-based
Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits warm-starting generalization loss in deep nets and proposes Hare & Tortoise, a fast-slow network pair where the fast Hare is periodically reset to the EMA-based slow Tortoise, improving generalization in warm-starting, continual learning, and Atari-100k RL.
Problem
Neural networks lose generalization ability after warm-starting on small, noisy data subsets, and existing trainability-preserving methods do not fix this, while full reinitialization discards useful knowledge.
Approach
A dual-network architecture inspired by complementary learning systems: a fast Hare network trained by gradient descent is periodically reset to a slow Tortoise network that tracks the Hare via exponential moving average, so forgetting preserves plasticity while the Tortoise retains generalizable knowledge.
Key results
- In warm-starting on CIFAR-100 with ViT-Tiny, Hare & Tortoise outperforms EMA, self-distillation and reinitialization baselines and exceeds fresh-network performance on CIFAR-10 with ResNet-18 (per the paper's claims).
- On Atari-100k with BBF, adding Hare & Tortoise (without self-supervised objective) improves IQM from 0.826 to 0.891 and OG from 0.397 to 0.372 while keeping compute unchanged.
- Standard trainability methods accelerate training loss reduction but do not improve test accuracy; Shrink & Perturb plus L2 and augmentation achieves the best test accuracy (e.g., 0.910 on CIFAR-10 ResNet-18 vs 0.924 for fresh network w/o warm-start).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Hare & Tortoise resets to DrQ | DrQ · Atari-100k | Median: 0.193 -> 0.260 | Helped |
| Reset the head every 20,000 steps instead of using the standard DrQ setup | DrQ · Atari-100k | IQM: 0.243 -> 0.332 | Helped |
| Add Hare & Tortoise resets to BBF with Shrink & Perturb and Head Reset | BBF · Atari-100k | IQM: 0.826 -> 0.891 | Helped |
| Add Hare & Tortoise resets to BBF with Shrink & Perturb and Head Reset | BBF · Atari-100k | OG: 0.397 -> 0.372 | Helped |
| Add the self-supervised objective instead of Hare & Tortoise resets to BBF with Shrink & Perturb and Head Reset | BBF · Atari-100k | IQM: 0.891 -> 0.940 | Helped |
| Add the self-supervised objective instead of Hare & Tortoise resets to BBF with Shrink & Perturb and Head Reset | BBF · Atari-100k | OG: 0.372 -> 0.377 | Hurt |
Surprising findings
- Modern networks warm-started on small noisy subsets retain near-perfect trainability yet lose generalization, contrary to the belief that trainability loss drives generalization loss.
- Common trainability-preserving methods (Spectral Decoupling, Regen, ReDo, CReLU) did not improve test accuracy despite faster training-loss reduction.
Limitations
- Hare & Tortoise was less effective than reinitialization on Tiny ImageNet with VGG-16, where generalization loss is severe.
- Momentum and reset hyperparameters were not tuned for the BBF integration; the self-supervised objective was intentionally excluded.
- Hare & Tortoise still relies on periodic hard resets in continual learning, which the authors note may be reducible.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100 Tiny ImageNet Atari-100k
Benchmarks: MNIST CIFAR-10 CIFAR-100 Tiny ImageNet Atari-100k
Metrics: test accuracy train accuracy IQM median mean optimality gap
Baselines: Shrink & Perturb Head Reset EMA Self-Distillation L2 regularization Data Augmentation Spectral Decoupling Regenerative Regularization ReDo CReLU DrQ BBF
Builds on: Shrink & Perturb DrQ BBF Ash & Adams warm-starting
Closest papers by meaning
- Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning (ICML 2025 · similarity 0.86)
- Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Rapid Learning without Catastrophic Forgetting in the Morris Water Maze (ICML 2024 · similarity 0.85)
- Neuroplastic Expansion in Deep Reinforcement Learning (ICLR 2025 · similarity 0.85)
- A mathematical theory of balancing relational generalization and memorization (NeurIPS 2026 · similarity 0.85)
- Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.