Research map / Reinforcement learning / Visual and representation RL
Towards Better Interpretability in Deep Q-Networks
Raghuram Mandyam Annasamy, Katia Sycara
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use directed uncertainty-based exploration instead of epsilon-greedy exploration | i-DQN vs DDQN · Atari games, 10M frames | training score: Alien: 1,533.45 -> 2,380.72; Freeway: 22.5 -> 28.79; Frostbite: 754.48 -> 3,968.45; Gravitar: 279.89 -> 517.33; MsPacman | Improved |
| Add reconstruction loss with weight 0.5 | i-DQN · MsPacman | training score: 3,245.3 | Hurt |
| Add reconstruction loss with weight 5.0 | i-DQN · MsPacman | training score: 3,013.1 | Hurt |
| Increase Bellman and distributional loss weights to 10.0 | i-DQN · MsPacman | training score: 5,267.15 | Hurt |
| Increase distributional-loss weight to 10.0 while keeping Bellman weight at 1.0 | i-DQN · MsPacman | training score: 4,916.0 | Hurt |
| Increase Bellman-loss weight to 10.0 while keeping distributional weight at 1.0 | i-DQN · MsPacman | training score: 4,053.6 | Hurt |
| Vary exploration factor across 0.1, 0.01, and 0.001 | i-DQN · Atari games | training score | No clear effect |
| Add distributional Bellman loss to diversity plus Bellman losses (ablation) | i-DQN · PongNoFrameskip-v4 | training reward | Mixed |
| Add reconstruction loss to diversity, Bellman, and distributional losses (ablation) | i-DQN · PongNoFrameskip-v4 | training reward | Mixed |
| Remove diversity loss (ablation) | i-DQN · PongNoFrameskip-v4 | attention diversity | Hurt |
| Use VAE (β=0) instead of AE for key reconstruction (ablation) | i-DQN · MsPacman (Color) | Agreement: 30.76 -> 29.78 | Hurt |
| Use VAE (β=0.01) instead of AE for key reconstruction (ablation) | i-DQN · MsPacman (Color) | Agreement: 30.76 -> 16.8 | Hurt |
Closest papers by meaning
- N$\text{A}^\text{2}$Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning (ICML 2023 · similarity 0.86)
- Frustratingly Easy Regularization on Representation Can Boost Deep Reinforcement Learning (CVPR 2023 · similarity 0.85)
- Graying the black box: Understanding DQNs (ICML 2016 · similarity 0.85)
- On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $\epsilon$-Greedy Exploration (NeurIPS 2023 · similarity 0.85)
- Q-SENN: Quantized Self-Explaining Neural Networks (AAAI 2024 · similarity 0.84)
- Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
