Research map / Reinforcement learning / Visual and representation RL

Hadamard Representation: Scaffolding Performance Across Model-free RL

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Learned state and action representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hadamard Representation (HR), a drop-in layer that multiplies two independently parameterized activation branches, to reduce tanh neuron dormancy and raise effective rank, and reports gains across DQN, PPO, PQN on Atari, SimbaV2 on HumanoidBench, and MR.Q on DMC-Visual.

Problem

Deep RL agents lose representational capacity during training through dormant neurons and effective rank collapse; existing remedies (resets, special architectures) are algorithm- or domain-specific.

Approach

Replace a hidden layer with the element-wise product of two independently parameterized activation branches. This lowers the per-neuron saturation probability for tanh (collapse requires both branches to saturate) and adds second-order feature interactions that raise effective rank at fixed width.

Key results

• On PQN across 51 Atari games, tanh + HR improves median human-normalized score over standard tanh by more than 100%.
• HR with SimbaV2 on HumanoidBench reaches 0.84 vs 0.71 baseline IQM (+18%); with MR.Q on DMC-Visual, 1.17 vs 1.00 (+17%).
• Parameter-matched widening degrades DQN to 0.49 (-46%) while HR reaches 1.03 (+13%) over baseline 0.91.

Surprising findings

• Naive widening of the DQN hidden layer to match HR's parameter count substantially degrades performance (-46%).
• HR with ReLU amplifies dormancy (+18%) and is therefore advised against for ReLU-based dormancy reduction, while tanh + HR reduces it (-23%).

Limitations

• Hyperparameters of baselines kept untuned, so HR-specific tuning could yield further gains but was not explored.
• Dormancy-reduction theory relies on assumed branch independence and symmetric saturation probabilities.
• Evaluation covers a subset of Atari games for DQN and PPO (8 games) and specific continuous control tasks.

Experimental setup

Datasets: Atari HumanoidBench DMC-Visual

Benchmarks: Atari HumanoidBench DMC-Visual

Metrics: baseline-normalized score median human-normalized score IQM success-normalized score dormant neuron fraction effective rank

Baselines: DQN PPO PQN SimbaV2 MR.Q CReLU ReDo DrQ-v2 XQC

Builds on: DQN PPO PQN SimbaV2 MR.Q cleanrl

Closest papers by meaning

• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning (ICLR 2025 · similarity 0.89)
• DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.85)
• Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
• DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.84)
• Hadamax Encoding: Elevating Performance in Model-Free Atari (NeurIPS 2025 · similarity 0.84)
• Learning Successor Features the Simple Way (NeurIPS 2024 · similarity 0.84)