Research map / Continual learning / Online and replay-based
Temporal-Difference Variational Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TD-VCL, variational continual learning objectives that regularize each posterior update against several past posterior estimates (n-step KL and TD(λ) forms), linking them to temporal-difference targets, and shows gains over VCL-family baselines on MNIST-style and CIFAR/TinyImageNet CL benchmarks.
Problem
Variational continual learning updates the posterior using only the most recent posterior estimate, so approximation errors compound over successive recursions and cause catastrophic forgetting.
Approach
Rewrite the VCL KL objective as a function of several past posterior estimates and task likelihoods, giving n-Step KL regularization and a TD(λ)-VCL objective with geometrically weighted past estimates; these are shown to equal averages or discounted sums of n-step TD targets.
Key results
- On PermutedMNIST-Hard at t=10, TD(λ)-VCL reaches 0.89 ±0.02 average accuracy vs 0.78 ±0.04 for VCL and 0.81 ±0.03 for VCL CoreSet.
- On SplitNotMNIST-Hard at t=5, n-Step TD-VCL and TD(λ)-VCL score 0.58 vs 0.51 for VCL and VCL CoreSet.
- On CIFAR100-10 at t=10, TD(λ)-VCL achieves 0.71 ±0.01 vs 0.66 for VCL; TD variants also improve UCL and UCB at later timesteps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TD(λ)-VCL instead of VCL | VCL · PermutedMNIST-Hard, t=10 | average accuracy: 0.78 ±0.04 -> 0.89 ±0.02 | Helped |
| Use n-Step TD-VCL instead of VCL | VCL · SplitMNIST-Hard, t=5 | average accuracy: 0.64 ±0.11 -> 0.67 ±0.04 | Helped |
| Use n-Step TD-VCL instead of VCL CoreSet | VCL · SplitNotMNIST-Hard, t=5 | average accuracy: 0.51 ±0.07 -> 0.58 ±0.08 | Helped |
| Use TD(λ)-VCL instead of VCL | VCL · CIFAR100-10, t=10 | average accuracy: 0.66 ±0.01 -> 0.71 ±0.01 | Helped |
| Use n-Step TD-VCL instead of VCL CoreSet | VCL · TinyImageNet-10, t=10 | average accuracy: 0.54 ±0.02 -> 0.56 ±0.02 | Helped |
| Add the TD(λ)-VCL objective to UCL | UCL · PermutedMNIST-Hard, t=10 | average accuracy: 0.73 ±0.12 -> 0.84 ±0.04 | Helped |
| Add the TD(λ)-VCL objective to UCB | UCB · SplitMNIST-Hard, t=5 | average accuracy: 0.75 ±0.10 -> 0.80 ±0.03 | Helped |
| Increase the number of considered previous posterior estimates n (ablation) | TD-VCL objectives · Continual-learning benchmarks | average accuracy | Mixed |
| Vary λ in TD(λ)-VCL (ablation) | TD(λ)-VCL · Continual-learning benchmarks | average accuracy | No clear effect |
| Vary likelihood-tempering parameter β (ablation) | TD-VCL objectives · Continual-learning benchmarks | average accuracy | Mixed |
Surprising findings
- Online and batch MLE baselines degrade sharply, while TD-VCL variants retain earlier tasks much better, with Task 1 accuracy around 80-85% versus 50-60% for VCL baselines.
Limitations
- Hyperparameters n and λ are setting-dependent and require tuning.
- Storing multiple past posterior estimates increases memory requirements.
- TD-VCL is not an RL algorithm; connections to MDP formalism are only analogical.
Experimental setup
Datasets: PermutedMNIST SplitMNIST SplitNotMNIST CIFAR100 TinyImageNet
Benchmarks: PermutedMNIST-Hard SplitMNIST-Hard SplitNotMNIST-Hard CIFAR100-10 TinyImageNet-10
Metrics: average accuracy across observed tasks per-task accuracy
Baselines: Online MLE Batch MLE VCL VCL CoreSet UCL UCB
Builds on: Variational Continual Learning UCB UCL TD(lambda)
Closest papers by meaning
- BECAME: Bayesian Continual Learning with Adaptive Model Merging (ICML 2025 · similarity 0.90)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off (AAAI 2025 · similarity 0.89)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.89)
- Learning to Continually Learn with the Bayesian Principle (ICML 2024 · similarity 0.89)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.