Research map / Reinforcement learning / Visual and representation RL
Simplicial Embeddings Improve Sample Efficiency in Actor–Critic Agents
Johan Obando-Ceron, Walter Mayor, Samuel Lavoie, Scott Fujimoto, Aaron Courville, Pablo Samuel Castro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add simplicial embeddings to the actor of FastTD3 (ablation) | FastTD3 · HumanoidBench (5 h1hand tasks) | aggregate return | Improved |
| add simplicial embeddings to the critic only (ablation) | FastTD3 · HumanoidBench (5 h1hand tasks) | aggregate return | Improved |
| replace SEM with Gumbel straight-through, vector quantization or C-ReLU on the actor encoder output (ablation) | FastTD3 actor · HumanoidBench (5 h1hand tasks) | aggregate return | Hurt |
| increase number of simplices L at fixed V (low-capacity to high-capacity regime) (ablation) | FastTD3 with SEM · HumanoidBench (5 h1hand tasks) | aggregate return | Mixed |
| increase simplex size V at fixed L (ablation) | FastTD3 with SEM · HumanoidBench (5 h1hand tasks) | aggregate return | Mixed |
| add SEM to the actor with fewer environments, smaller replay buffer or smaller batch size (ablation) | FastTD3 · HumanoidBench (5 h1hand tasks) | aggregate return | Improved |
| add SEM to the actor when CDQ or C51 are removed (ablation) | FastTD3 · HumanoidBench (5 h1hand tasks) | aggregate return | Improved |
| add SEM to the actor of PPO | PPO (CleanRL) · Atari (28 ALE games) | IQM return | Improved |
| add SEM to the actor of PPO | PPO (CleanRL) · Isaac Gym | return | Improved |
| add SEM to FastTD3, FastTD3-SimBaV2 and FastSAC actors | FastTD3, FastTD3-SimBaV2, FastSAC · HumanoidBench (28 h1hand tasks) | return | Improved |
| add SEM to FastTD3 | FastTD3 · MTBench MT50 | return | Improved |
| add SEM to the flow-based actor of FQL | FQL · OGBench (3 tasks, offline-to-online) | return | Improved |
Closest papers by meaning
- Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learning (ICLR 2025 · similarity 0.87)
- Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning (ICML 2024 · similarity 0.86)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
- SiT: Symmetry-invariant Transformers for Generalisation in Reinforcement Learning (ICML 2024 · similarity 0.86)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
