Research map / Reinforcement learning / Visual and representation RL

The Laplacian Keyboard: Beyond the Linear Span

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Probabilistic and autonomous RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes the Laplacian Keyboard, a hierarchical meta-policy that sequences USFA-induced behaviors over Laplacian eigenvector weights to go beyond the zero-shot linear span, evaluated on DMC and Item-Collector.

Problem

Zero-shot RL with Laplacian eigenvector representations is limited to rewards within the linear span of the chosen eigenvectors, so policies can be suboptimal for tasks outside that span.

Approach

Treat USFA policies over Laplacian eigenvector weights as a task-agnostic library of options, and train a meta-policy that outputs a weight vector every fixed horizon to stitch behaviors dynamically, going beyond the linear span without reward features.

Key results

• On Cheetah with k=1, LK improves over zero-shot by more than 100% on several tasks (e.g., Run-B +25129% relative improvement reported in table).
• On Quadruped, LK's downstream AUC ratio vs TD3 reaches at least 2.4x across all basis sizes.
• On Item-Collector, LK-USFA performs close to the privileged OKB baseline without reward features, and LK-GPE&GPI lags at larger k.

Surprising findings

• Quadruped achieves its best zero-shot performance at k=1, and scaling the Laplacian basis size yields no consistent gains.
• Zero-shot Laplacian performance is statistically indistinguishable from FB at k=50 on APS, despite FB being the stronger-theory baseline.

Limitations

• Options are terminated after a fixed horizon rather than with learned termination conditions.
• Empirical improvements over zero-shot are mixed on some tasks (e.g., Quadruped and some Cheetah/Walker tasks at larger k).
• Evaluation limited to three DMC domains and one small gridworld.

Experimental setup

Datasets: DMC offline datasets (APS, Proto, RND) Item-Collector reward-free uniform random walk data

Benchmarks: DeepMind Control Suite (Cheetah, Quadruped, Walker) Item-Collector

Metrics: undiscounted return relative improvement over zero-shot (%) AUC ratio vs TD3

Baselines: Forward-Backward (FB) TD3 Option Keyboard Basis (OKB) LK-GPE&GPI

Builds on: Proto-Value Functions Universal Successor Feature Approximators Option Keyboard Augmented Laplacian Objective (ALLO) Forward-Backward representations TD3

Closest papers by meaning

• Online Laplacian-Based Representation Learning in Reinforcement Learning (ICML 2025 · similarity 0.89)
• Proper Laplacian Representation Learning (ICLR 2024 · similarity 0.89)
• Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.86)
• Constructing an Optimal Behavior Basis for the Option Keyboard (NeurIPS 2025 · similarity 0.86)
• Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.86)
• Model-free Low-Rank Reinforcement Learning via Leveraged Entry-wise Matrix Estimation (NeurIPS 2024 · similarity 0.86)