Research map / Reinforcement learning / Visual and representation RL

Spatial-Aware Decision-Making with Ring Attractors in Reinforcement Learning Systems

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: Inferring decision-making from behavior · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Integrates ring attractor networks as a circular action-selection mechanism into RL agents (exogenous CTRNN-based and RNN-based DL module), reporting a 53% mean improvement over EfficientZero on Atari 100k.

Problem

Action selection in RL ignores topological relations between actions (actions are treated as orthogonal outputs), so spatial structure and uncertainty are not exploited in decision-making, slowing learning in spatially structured tasks.

Approach

Map actions to positions on a ring, set each action's Q-value as the amplitude of a Gaussian input to a Touretzky-style ring attractor (CTRNN) or a ring-structured RNN output layer, with Bayesian linear regression providing per-action variance; the decoded attractor bump gives the selected action.

Key results

• On Atari 100k, EffZeroRA reaches a human-normalised mean of 1.454 vs 0.959 for re-run EfficientZero (about 53% average improvement claimed).
• Largest gains reported on Asterix (31037.3 vs 14839.3) and Boxing (62.4 vs 30.5), roughly 110% and 105% improvements.
• CTRNN exogenous ring attractor models incur a mean computational overhead of 297.3% over the baseline, motivating the DL implementation.

Measured effects reported in the paper

Surprising findings

• Exogenous CTRNN ring attractor incurred roughly 297% computational overhead, motivating a DL reimplementation.
• Random (incorrect) action placement on the ring degraded performance, supporting that topology matters.

Limitations

• Evaluation is limited to discrete or 1-D circular action spaces; continuous control and multi-agent settings are not tested.
• Reported 53% gain is relative to a re-run EfficientZero baseline and averaged over games; mean improvement claims rely on a single comparison set.
• Ring topology may offer little benefit when the action count is small (|A|<3) or actions lack spatial relations.

Experimental setup

Datasets: Atari 100k OpenAI Gym Super Mario Bros OpenAI highway

Benchmarks: Atari 100k OpenAI Gym Super Mario Bros OpenAI highway

Metrics: game score human-normalised mean score human-normalised median score cumulative reward mean episodic return

Baselines: EfficientZero SPR CURL BDQN DDQN PPO

Builds on: EfficientZero BDQN DDQN PPO Touretzky ring attractor model

Closest papers by meaning

• Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments (NeurIPS 2025 · similarity 0.85)
• DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.84)
• A Multi-Region Brain Model to Elucidate the Role of Hippocampus in Spatially Embedded Decision-Making (ICML 2025 · similarity 0.84)
• Deep Reinforcement Learning with Time-Scale Invariant Memory (AAAI 2025 · similarity 0.84)
• Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning (ICML 2026 · similarity 0.83)
• Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning (ICML 2025 · similarity 0.83)