Research map / Multi-agent systems and games / Cooperative multi-agent RL
eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes eQMARL, a distributed quantum actor-critic where a split quantum critic spread across agents is coupled via Bell-type entangled input qubits and joint measurement, eliminating local observation sharing in QMARL.
Problem
Cooperation in distributed multi-agent RL normally requires sharing local observations or classical communication; existing quantum MARL (QMARL) methods coordinate via classical communication and do not exploit entanglement over a quantum channel.
Approach
A quantum critic split across agents as local VQC branches, with input qubits pairwise entangled by a central source (Bell-state variants, e.g. Psi+) and a joint Z-basis measurement estimating the joint value; agents update their branches with partial gradients from a single classical scalar, so observations are never shared.
Key results
- Psi+ entanglement converges 17.8% faster to score 25 than qfCTDE in CoinGame-2 POMDP and is 10.8% faster than sCTDE in MDP.
- eQMARL uses 1 central trainable parameter versus 25 central parameters in sCTDE (25-times fewer centralized parameters).
- In MiniGrid, eQMARL reached the goal in as few as 9 steps versus 17 for fCTDE; qfCTDE and sCTDE failed within 50 steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Ψ+ input entanglement instead of no entanglement (ablation) | eQMARL · CoinGame-2 POMDP | convergence speed to score 25: 10.7% faster | Helped |
| use Ψ+ input entanglement instead of Ψ− (ablation) | eQMARL · CoinGame-2 MDP | convergence speed to score 20: 4.5% faster | Helped |
| use Ψ+ input entanglement instead of no entanglement (ablation) | eQMARL · CoinGame-2 POMDP | convergence speed to score 20: 2% faster | Helped |
| use Φ+ input entanglement instead of Ψ− (ablation) | eQMARL · CoinGame-2 MDP | convergence speed to score 25: 5.2% faster | Helped |
| use Φ+ or Φ− input entanglement instead of Ψ+ (ablation) | eQMARL · CoinGame-2 POMDP | reaching score 25 | Hurt |
| increase VQC depth from L=2 to L=5 (ablation) | eQMARL · CoinGame-2 MDP | mean score: 3-times higher | Helped |
| increase VQC depth from L=5 to L=10 (ablation) | eQMARL · CoinGame-2 MDP | mean score: nearly identical | No clear effect |
| use VQC depth L=5 instead of L=2 (ablation) | qfCTDE · CoinGame-2 | score | Helped |
| increase classical hidden units from h=6 to h=12 (ablation) | fCTDE and sCTDE · CoinGame-2 | score: nearly 2-times | Helped |
| increase classical hidden units from h=12 to h=24 (ablation) | fCTDE and sCTDE · CoinGame-2 | score: only marginally less | No clear effect |
| use eQMARL-Ψ+ instead of qfCTDE | CoinGame-2 · CoinGame-2 MDP | convergence speed to score 20: 16.2% faster | Helped |
| use eQMARL-Ψ+ instead of sCTDE | CoinGame-2 · CoinGame-2 MDP | maximum score: 1.4% increase | Helped |
Surprising findings
- Phi+ and Phi- entanglement (same-state coupling) performed worse than Psi+ and often no input entanglement, in contrast to expectations of stronger coupling.
- Under POMDP the quantum models required more total parameters than classical counterparts due to classical dimensionality-reduction networks.
Limitations
- Quantum circuits simulated classically (tensorflow-quantum/cirq), with substantial training time overhead.
- Assumes ideal, lossless quantum channel and trusted central source distributing entangled qubits.
- Policy sharing across agents and a classical reward channel are assumed; results limited to small environments (CoinGame-2, small CartPole and MiniGrid).
Experimental setup
Datasets: CoinGame-2 multi-agent CartPole multi-agent MiniGrid
Benchmarks: CoinGame-2 multi-agent CartPole multi-agent MiniGrid
Metrics: score convergence speed to score threshold total/average reward trainable parameter count
Baselines: fCTDE sCTDE qfCTDE eQMARL with None/Phi+/Phi-/Psi- entanglement
Builds on: MAA2C CTDE split learning Bell state entanglement QMARL (Chen et al.)
Closest papers by meaning
- Exponential Quantum Communication Advantage in Distributed Inference and Learning (NeurIPS 2024 · similarity 0.85)
- Multi-agent Markov Entanglement (NeurIPS 2025 · similarity 0.84)
- Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis (NeurIPS 2026 · similarity 0.84)
- Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach (ICML 2025 · similarity 0.84)
- Inverse Factorized Soft Q-Learning for Cooperative Multi-agent Imitation Learning (NeurIPS 2024 · similarity 0.84)
- Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.