AI Paper Atlas

Research map / Continual learning / Online and replay-based

Permanent and Transient Representations for Continual Reinforcement Learning

Nishanth Anand, Doina Precup

NeurIPS 2026 · Continual learning › Online and replay-based · approach: Plasticity-preserving continual reinforcement learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a continual RL method that pairs permanent and transient representations as complementary learning systems to address the stability-plasticity trade-off, extending Anand & Precup (2023), which used permanent and transient value functions.

Problem

Continual reinforcement learning agents must adapt to new situations while retaining prior knowledge, facing a stability-plasticity trade-off.

Approach

Applies the complementary learning systems idea at the representation level, using separate permanent (slow, long-term) and transient (fast, adaptive) representations rather than only permanent and transient value functions as in Anand & Precup (2023).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use separate (first-order permanent, higher-order transient) features instead of all features in both PT-TD components (ablation)PT-TD · 2D grid prediction (linear Fourier features)RMSVEImproved
restrict permanent features to first-order only in PT-TD (separate) vs using both orders (ablation)PT-TD · 1D chain predictionRMSVE after task changeImproved
use only one feature order in plain TD learning (ablation)TD learning · 1D chain predictionRMSVEHurt
use a single reward-correlated feature for the transient Q function (separate variant) (ablation)PTQ-learning · 7-state chain controladaptation speed: re-adjusts within five episodes after task changeImproved
reset weights at the start of each task (reset Q-learning) vs continualQ-learning · 7-state chain controlepisodic returnHurt
add EWC or continual backprop to PQNPQN · CIFAR image-task MDPreturnNo clear effect
use fully neural-network transient component (PT-NN) instead of non-parametric transient (ablation)PT · CIFAR image-task MDPreturnHurt
use only the non-parametric transient approximator (T-Only), no permanent network (ablation)PT · CIFAR image-task MDPreturnHurt
pair neural permanent value function with MinHash non-parametric transient tablePQN · CIFAR image-task MDP (hidden task boundaries)adaptation after task changeImproved
pair PQN with MinHash transient tablePQN(32) · Craftax-Classic 250Msteps to return 15: over 200M -> 150M stepsImproved
use MinHash instead of random hashing in the transient table (ablation)transient table · symbolic observationsvalue correction qualityImproved

Experimental setup

Builds on: Anand & Precup (2023) permanent and transient value functions

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy