Research map / Reinforcement learning / Visual and representation RL
The Curse of Diversity in Ensemble-Based Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that training a diverse ensemble of data-sharing RL agents can degrade each member versus single-agent training, attributes this to a low share of self-generated data in the shared buffer, and proposes a fix.
Problem
Data-sharing ensembles of diverse RL agents, a common exploration strategy, can hurt individual member performance compared with standard single-agent training.
Approach
Attributes the degradation to the low proportion of self-generated data in the shared replay buffer (the 'curse of diversity'), as diversity makes members learn from each other's off-policy data, and proposes a remedy to restore self-generated data share.
Key results
- Diverse data-sharing ensembles can significantly impair individual member performance relative to standard single-agent training, per the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train diverse agents with shared data instead of standard single-agent training | Ensemble-based exploration · Deep reinforcement learning | individual-agent performance | Hurt |
Surprising findings
- Training a diverse ensemble of data-sharing agents, a well-established exploration strategy, can degrade individual members below single-agent performance.
Experimental setup
Baselines: single-agent training data-sharing ensemble
Builds on: ensemble-based exploration off-policy deep RL
Closest papers by meaning
- Diverse Projection Ensembles for Distributional Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Enhancing Diversity In Parallel Agents: A Maximum State Entropy Exploration Story (ICML 2025 · similarity 0.86)
- Scalable Exploration via Ensemble++ (NeurIPS 2025 · similarity 0.85)
- Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.85)
- Constrained Ensemble Exploration for Unsupervised Skill Discovery (ICML 2024 · similarity 0.85)
- Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.