Research map / Reinforcement learning / Policy gradient and actor-critic
Functional Regularization for Reinforcement Learning via Learned Fourier Features
Alexander Cong Li, Deepak Pathak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MLP with LFF architecture | SAC · DMControl state-based (8 envs) | return / sample efficiency: better in 6 out of 8 environments, slightly better in the remaining 2 | Improved |
| replace CNN with CNN+LFF | SAC + RAD · DMControl pixels (4 envs) | sample efficiency | Improved |
| add LFF to PPO policy and critic | PPO · OpenAI Gym (4 envs) | sample efficiency | No clear effect |
| use LFF for only the critic | PPO · OpenAI Gym (4 envs) | return | Mixed |
| use LFF only for Q-network (ablation) | SAC · Walker, Quadruped | return | Improved |
| use LFF only for policy network (ablation) | SAC · Walker, Quadruped | return | No clear effect |
| fix Fourier basis B instead of training it (ablation) | LFF · DMControl 4 envs | return | Hurt |
| remove input concatenation (ablation) | LFF · Quadruped, Humanoid | return | Hurt |
| remove input concatenation (ablation) | LFF · Walker, Finger | return | No clear effect |
| use log-uniform initialization of B instead of N(0, sigma^2) (ablation) | LFF · DMControl | return | Hurt |
| concatenate input x to MLP first layer output (ablation) | MLP · state-based DMControl | return | No clear effect |
| remove target network | SAC with LFF vs MLP · DMControl state-based | return | Improved |
Closest papers by meaning
- Control Regularization for Reduced Variance Reinforcement Learning (ICML 2019 · similarity 0.90)
- Fourier Policy Gradients (ICML 2018 · similarity 0.88)
- Overcoming The Spectral Bias of Neural Value Approximation (ICLR 2022 · similarity 0.88)
- Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning (ICML 2024 · similarity 0.86)
- Parseval Regularization for Continual Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Fourier Guided Adaptive Adversarial Augmentation for Generalization in Visual Reinforcement Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
