Research map / Bandits and online learning / RL theory and MDPs
Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Bound
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a Frobenius-norm confidence ball instead of the 2-1 norm ball, under stronger feature regularity (ablation) | UC-MatrixRL · Online episodic MDP, feature space | regret upper bound: O[√(Cψ(CM + {C′ψ}²) · ln(CφT)] · H² · √(d³T) -> O[√(Cψ(CM + {C′ψ}²) · ln(CφT)] · dH² · √T | Helped |
| Represent features through kernel functions rather than explicit feature vectors (ablation) | MatrixRL · Online episodic MDP, kernel space | regret upper bound: O(Cψ · ||P||Hφ×Hψ · log(T) · d̃ · H² · √T) | Helped |
| Replace maximization over the confidence ball with the stated closed-form confidence bound (ablation) | UC-MatrixRL · Online episodic MDP, feature space | regret upper bound | No clear effect |
| Add optimistic reward estimation when rewards are unknown and linearly parameterized (ablation) | MatrixRL · Online episodic MDP with unknown linear rewards | regret: an extra Õ(d√T) regret | Hurt |
Closest papers by meaning
- Kernel-Based Reinforcement Learning: A Finite-Time Analysis (ICML 2021 · similarity 0.91)
- Model-based Reinforcement Learning for Continuous Control with Posterior Sampling (ICML 2021 · similarity 0.90)
- Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm (NeurIPS 2024 · similarity 0.89)
- Reward-Free Kernel-Based Reinforcement Learning (ICML 2024 · similarity 0.89)
- No-Regret Reinforcement Learning in Smooth MDPs (ICML 2024 · similarity 0.88)
- Hybrid Reinforcement Learning Breaks Sample Size Barriers In Linear MDPs (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.