Research map / Reinforcement learning / Visual and representation RL
Benchmarking Quantum Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a statistical benchmarking methodology for (quantum) RL based on an empirical sample-complexity estimator and a significance-based definition of outperformance, applies it to a new 6G beam-management environment and CartPole, and finds quantum models competitive with but not clearly superior to parameter-matched classical models.
Problem
Benchmarking RL and quantum RL is hard: heuristic algorithms lack performance guarantees, results are sensitive to randomness, and small-seed studies make claims of quantum advantage in sample complexity unreliable.
Approach
An empirical sample-complexity estimator defined relative to a performance threshold (1-ε) and probability threshold δ over N=100 training runs, plus a significance criterion (non-overlapping 5th–95th percentile ranges via cluster resampling) for declaring outperformance; applied to hybrid VQC–classical DDQN/PPO versus classical networks on a scalable beam-management environment.
Key results
- Hybrid 14-qubit VQC DDQN (437 variational params) consistently has lower empirical sample complexity than a parameter-matched classical DNN (387 params) and is competitive with the 4611-parameter classical DNN.
- Classical sample complexity saturates at hidden width 64, while the quantum model does not saturate up to 14 qubits; the largest classical model is not significantly outperformed by the large quantum model.
- On CartPole-v1 the best quantum setup (single layer, 4 qubits) performs competitively with the best classical model; the gap is often not statistically significant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a hybrid quantum model instead of a similarly sized classical DNN | DDQN · BeamManagement6G | sample complexity | Helped |
| Use a 14-qubit hybrid model instead of the width-64 classical DNN | DDQN · BeamManagement6G | sample complexity | Mixed |
| Increase the classical DNN width from 16 to 64 (ablation) | DDQN · BeamManagement6G | sample complexity | Helped |
| Use the 14-qubit hybrid model instead of the 10-qubit hybrid model (ablation) | DDQN · BeamManagement6G | sample complexity | Helped |
| Increase classical DNN width beyond 64 (ablation) | DDQN · BeamManagement6G | sample complexity | No clear effect |
| Increase the quantum model's qubit count up to 14 (ablation) | DDQN · BeamManagement6G | sample complexity | Helped |
| Use PPO instead of DDQN | BeamManagement6G RL agents · BeamManagement6G | sample complexity | Hurt |
| Use a hybrid quantum function approximator instead of a classical one | PPO · BeamManagement6G | sample complexity | Helped |
| Use a four-qubit quantum model instead of the best classical model | Vanilla policy gradient · CartPole-v1 | sample complexity | No clear effect |
| Stack multiple past observations (ablation) | BeamManagement6G agents · BeamManagement6G | overall performance | No clear effect |
| Use trained RL policies instead of a random strategy | Classical and quantum RL policies · BeamManagement6G | cumulative intensity | Helped |
Surprising findings
- Claimed quantum advantage in prior QRL work largely disappears when runs are increased to 100 seeds and significance is tested with percentile ranges.
- Small fully classical networks are not competitive with hybrid models at equal parameter count, yet the largest classical model matches the quantum model.
Limitations
- Noise-free simulation only; hardware noise and shot noise not modeled.
- Hyperparameter search cannot guarantee the best classical model was found, so quantum advantage cannot be concluded.
- Quantum models limited to 14 qubits due to simulation cost, so classical simulability and scaling behavior remain untested.
Experimental setup
Datasets: BeamManagement6G (introduced, simulated) CartPole-v1
Benchmarks: BeamManagement6G CartPole-v1
Metrics: empirical sample complexity 5th-95th percentile ranges (cluster resampling) cumulative beam intensity relative to optimum
Baselines: classical DNN (width 16, 32, 64) classical PPO vanilla policy gradient classical
Builds on: double deep Q-learning proximal policy optimization vanilla policy gradient cluster resampling PennyLane
Closest papers by meaning
- Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach (ICML 2025 · similarity 0.88)
- Position: Benchmarking is Limited in Reinforcement Learning Research (ICML 2024 · similarity 0.88)
- QCircuitBench: A Large-Scale Dataset for Benchmarking Quantum Algorithm Design (NeurIPS 2025 · similarity 0.86)
- TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search (NeurIPS 2025 · similarity 0.86)
- Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret (ICML 2024 · similarity 0.86)
- Reinforcement Learning for Quantum Control under Physical Constraints (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.