Research map / Reinforcement learning / Visual and representation RL
Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHNN, a hierarchical subequivariant RL architecture that uses greedy bipartite task assignment and entity-level subequivariant message passing to build local reference frames for 3D multi-entity morphology control, and introduces the MeBen benchmark.
Problem
Policy learning for multi-entity systems in 3D physical environments suffers from exponential growth of the global state space; existing morphology-based RL benchmarks and methods lack multi-entity, symmetry-aware settings.
Approach
Dynamically decouple the global space into entity-level local graphs via greedy bipartite task assignment, then apply gravity-subequivariant message passing to learn per-entity local reference frames (via orthogonalization) replacing hand-crafted heading normalization.
Key results
- On Team Reach, SHNN reaches 97.26% success on 1_ant versus 93.39% for MLP+HN, and 77.93% on 2_ants versus 5.25% for MLP+HN.
- On the harder 2_ant_claw_centipede setting, SHNN achieves 8.70% vs 3.98% for MLP+HN (low absolute success).
- SHTransformer improves over Transformer+HN on most Team Reach settings, e.g., 63.61% vs 5.47% on 1_ant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHNN instead of MLP+HN | Team Reach policy · Team Reach: 2_ants | Success Rate (%): 5.25 ± 1.399 -> 77.93 ± 22.22 | Helped |
| Apply SHNN plug-in to Transformer control instead of Transformer+HN | Transformer policy · Team Reach: 2_ant_claw | Success Rate (%): 0.51 ± 0.19 -> 1.17 ± 0.70 | Helped |
| Use greedy task assignment instead of stochastic or no assignment (ablation) | SHNN · Team Reach | Success Rate (%) | Helped |
| Remove task assignment and use a fully connected entity graph (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Use stochastic assignment instead of greedy bipartite matching (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Remove gravity awareness by using SO(3) instead of gravity-conditioned SO_g(3) (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Use O_g(3) rather than SO_g(3) equivariance (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Replace subequivariant message passing with goal-direction hand-crafted LRF (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Replace subequivariant message passing with heading-direction hand-crafted LRF (ablation) | SHNN · Team Reach | Success Rate (%) | Hurt |
| Remove translation invariance (ablation) | SHNN · Team Reach: 3_ant_claw_centipede | Success Rate (%) | Hurt |
| Use sparse root-mean subtraction without decoupling translation transformations (ablation) | SHNN · Team Reach: 3_ant_claw_centipede | Success Rate (%) | Hurt |
| Use dense body-mean subtraction without decoupling translation transformations (ablation) | SHNN · Team Reach: 3_ant_claw_centipede | Success Rate (%) | Hurt |
Surprising findings
- Task assignment yields limited improvement in Team Sumo, where equivariance matters more.
- SHGNN outperforms SHNN on 3_ant_claw_centipede (31.07% vs 8.70%) despite lower success on 2_ants, while training more slowly.
Limitations
- Assignment is rule-based greedy matching rather than learnable, and the paper calls for co-learnable assignment and equivariance.
- Relies on state-based inputs; vision-based inputs not addressed.
- Symmetry assumption limited to E_g(3) classical physics; not applicable to non-Euclidean environments.
Experimental setup
Datasets: MeBen (Team Reach, Team Sumo) MxT-Bench morphologies unimals
Benchmarks: MeBen Team Reach MeBen Team Sumo
Metrics: success rate win rate
Baselines: MLP MLP+HN Transformer+HN HNN+DN HNN+HN SHGNN
Builds on: MAPPO MxT-Bench Brax heading normalization
Closest papers by meaning
- Subequivariant Graph Reinforcement Learning in 3D Environments (ICML 2023 · similarity 0.87)
- Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion (ICLR 2026 · similarity 0.84)
- Hierarchical Equivariant Policy via Frame Transfer (ICML 2025 · similarity 0.84)
- Neural Production Systems (NeurIPS 2021 · similarity 0.84)
- Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance (NeurIPS 2024 · similarity 0.82)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.