Research map / Reinforcement learning / Visual and representation RL

SMoSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks

AAAI 2025 · Reinforcement learning › Visual and representation RL · approach: In-context and adaptive RL · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SMoSE, a top-1 sparse mixture of linear (shallow) experts with a linear router trained end-to-end with SAC plus load-balancing losses, distilling decision trees from the router for interpretability on six MuJoCo tasks.

Problem

Interpretable reinforcement learning for continuous control: interpretable policies usually underperform closed-box neural policies in high-dimensional MuJoCo tasks.

Approach

A top-1 sparse Mixture-of-Experts whose experts and router are both shallow linear functions, trained jointly via SAC with importance and load-balancing penalties, so only one interpretable linear expert controls each state; decision trees are distilled from the router for readability.

Key results

• SMoSE achieves the best average episodic return among interpretable methods on five of six MuJoCo tasks, e.g., 4224.29 on Walker2d-v4 vs 1090 for CGP (values for CGP/LGP/Metric-40 visually derived from plots).
• SMoSE on Hopper-v4 reaches 2816.08 with 168 active parameters (672 total), close to SAC-S (3076.09 with 186 active).
• SMoSE underperforms on Swimmer-v4 (45.40 vs 280 for CGP), and SAC-L generally achieves the highest returns among non-linear baselines.

Measured effects reported in the paper

Surprising findings

• On Swimmer-v4 SMoSE does not outperform any interpretable method nor PPO, and all SAC-based policies perform weaker there.

Limitations

• Performance gap remains with non-interpretable SAC-L, especially on Ant-v4, HalfCheetah-v4, and Reacher-v4.
• Swimmer-v4 is a failure case where SMoSE does not beat interpretable baselines or PPO.
• Interpretation of experts requires manual analysis; distilled decision trees are only a support and not exact.

Experimental setup

Datasets: MuJoCo Walker2d-v4 Hopper-v4 Ant-v4 HalfCheetah-v4 Reacher-v4 Swimmer-v4

Benchmarks: Walker2d-v4 Hopper-v4 Ant-v4 HalfCheetah-v4 Reacher-v4 Swimmer-v4

Metrics: average episodic return active and total parameter count

Baselines: CGP LGP Metric-40 PPO SAC-S SAC-M SAC-L

Builds on: Soft Actor-Critic sparse mixture of experts CART decision trees

Closest papers by meaning

• Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer (ICML 2025 · similarity 0.87)
• ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policies (NeurIPS 2026 · similarity 0.86)
• Mixture-of-Experts Meets In-Context Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.85)
• Score Models for Offline Goal-Conditioned Reinforcement Learning (ICLR 2024 · similarity 0.85)
• Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts (ICLR 2024 · similarity 0.84)