Research map / Reinforcement learning / Visual and representation RL
Adaptive Rational Activations to Boost Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes learnable rational activation functions (and a weight-shared joint-rational variant) in deep RL, showing they boost plasticity and let DQN match DDQN and Rainbow on Atari.
Problem
Deep RL agents suffer from limited neural plasticity under distribution shifts, yet most approaches use static activation functions; the paper asks whether learnable activations can augment plasticity and reduce overestimation.
Approach
Use rational activation functions (ratio of polynomials with learnable coefficients) per layer, prove that rationals with numerator degree greater than denominator degree embed residual connections, and propose joint-rationals that share one set of rational parameters across all layers as a natural regulariser.
Key results
- Rational DQN and joint-rational DQN reach super-human performance on 11 of 15 Atari games versus 3/15 for DQN with Leaky ReLU (table of normalised scores).
- DQN with rational plasticity outperforms rigid DDQN on every considered Atari game, making DQN competitive with DDQN and Rainbow.
- Rigid baselines need up to 3.5 times as many parameters as rational-activation networks to match performance on VGG classification experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rational activations instead of fixed activations in DQN | DQN · Atari games | performance | Helped |
| Share rational activation parameters across layers (joint-rational) (ablation) | DQN · Atari games | mean score: 6/15 -> 9/15 best mean-score wins | Helped |
| Use rational activations instead of Leaky ReLU | Rainbow · Atari games | final mean score | Helped |
| Share rational activation parameters across layers (ablation) | Rainbow · Atari games | final mean score | Mixed |
| Use rational activations instead of Leaky ReLU | VGG4 · CIFAR 10 | Testing Acc@1: 80.0±1. -> 84.3±.5 | Helped |
| Use rational activations instead of Leaky ReLU | VGG4 · CIFAR 100 | Testing Acc@1: 56.5±.9 -> 58.9±.6 | Helped |
| Replace a lesioned residual block with a rational activation (ablation) | ResNet101 · ResNet lesioning (testing) | recovery (%): 93.1, 97.1, 81.6, 81.7 -> 90.5, 97.6, 91.5, 85.3 | Mixed |
| Freeze the learned rational activation functions before another training session (ablation) | DQN with rational activations · Dynamic and progressive Atari environments | performance | Hurt |
| Use PELU instead of rational activations (ablation) | DQN · Atari games | super-human performances: 3 out of 9 -> 7 out of 9 games | Hurt |
| Use SiLU-family activations instead of Leaky ReLU (ablation) | DQN · Atari games | performance | No clear effect |
| Use rational activations instead of Leaky ReLU in DQN | DQN · Atari games | overestimation | Helped |
| Replace Leaky ReLU with rational activations in VGG4 (ablation) | VGG4 · CIFAR 10 | Testing Acc@1 | Helped |
Surprising findings
- Rainbow with plasticity is better than rigid Rainbow, but rigid Rainbow never outperforms DQN with neural plasticity, and Rainbow lost to vanilla DQN on 3 games.
- Tempered (frozen) rational activations perform better than full plasticity on some stationary environments, suggesting extra plasticity can hurt generalisation.
Limitations
- Rational degrees (5,4) were chosen empirically and not automatically selected
- Rainbow results are from single runs only due to computational cost
- Results limited to 15 Atari games with 5 seeds for DQN/DDQN
Experimental setup
Datasets: Atari 2600 (15 games) MNIST Permuted MNIST CIFAR-10 CIFAR-100
Benchmarks: Atari 2600 (15 games) Permuted MNIST CIFAR-10 CIFAR-100
Metrics: human-normalised mean score superhuman probability relative overestimation classification accuracy
Baselines: DQN with Leaky ReLU DDQN with Leaky ReLU Rainbow with Leaky ReLU DQN with SiLU DQN with SiLU+dSiLU PELU CReLU
Builds on: DQN Double DQN Rainbow Padé activation units PELU CReLU
Closest papers by meaning
- Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages (ICLR 2024 · similarity 0.86)
- A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning (ICML 2025 · similarity 0.86)
- Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning (NeurIPS 2025 · similarity 0.85)
- Neuroplastic Expansion in Deep Reinforcement Learning (ICLR 2025 · similarity 0.85)
- Adaptive $Q$-Network: On-the-fly Target Selection for Deep Reinforcement Learning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.