Research map / Reinforcement learning / Visual and representation RL

Spectral Alignment in Forward–Backward Representations via Temporal Abstraction

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Learned state and action representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that action repetition (temporal abstraction) acts as a spectral low-pass filter on the successor representation, reducing its effective rank and improving forward-backward (FB) representation learning in continuous control.

Problem

Forward-backward representations assume a low-rank successor representation, but continuous environments yield high-rank, slowly decaying SR spectra; this spectral mismatch makes FB learning unstable, especially at high discount factors.

Approach

Action repetition replaces the one-step transition operator with its k-step counterpart, which accelerates spectral decay of the SR and yields a lower-rank target for FB learning; the paper pairs this with an optimality-gap bound that trades action-repeat error against spectral truncation error.

Key results

• Adding action repetition (k>1) boosts episodic return in continuous four-rooms, whereas increasing embedding dimension d or discount gamma alone with k=1 does not improve performance.
• Increasing embedding dimension or discount without temporal abstraction increases Bellman error; increasing k does not increase Bellman error.
• Performance is highest for moderate action-repeat k in [5,10] combined with moderate gamma and d, with lower nominal gamma plus larger k outperforming standard high-discount settings.

Surprising findings

• Increasing FB embedding dimension from 25 to 400 yields no performance gain without temporal abstraction in continuous settings.
• Excessive action repetition degrades performance, as stable rank and entropy approach minima and task-relevant dynamics are lost.

Limitations

• Theoretical analysis restricted to finite state and action spaces; continuous-setting theory not provided.
• Focuses on action repetition only; options or learned skills not analyzed beyond a preliminary action chunking comparison.
• Introduces bias that limits resolution of high-frequency dynamics, less suitable for reactive control.

Experimental setup

Datasets: OGBench Four-Rooms OGBench Maze OGBench Large-Maze OGBench Cube OGBench Scene

Benchmarks: OGBench Four-Rooms OGBench Maze OGBench Large-Maze OGBench Cube OGBench Scene

Metrics: episodic return stable rank normalized spectral entropy Bellman error

Baselines: FB with k=1 (no action repetition) action chunking

Builds on: forward-backward representation successor representation OGBench

Closest papers by meaning

• Learning Robust Spectral Dynamics for Temporal Domain Generalization (NeurIPS 2025 · similarity 0.85)
• SpectraLDS: Provable Distillation for Linear Dynamical Systems (NeurIPS 2025 · similarity 0.85)
• Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures (NeurIPS 2026 · similarity 0.85)
• MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models (NeurIPS 2026 · similarity 0.85)
• Latent Functional Maps: a spectral framework for representation alignment (NeurIPS 2024 · similarity 0.84)
• ResKoopNet: Learning Koopman Representations for Complex Dynamics with Spectral Residuals (ICML 2025 · similarity 0.84)