Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning for Quantum Control under Physical Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates pulse-level quantum control (Λ system, two-photon Rydberg gate, transmon reset) as a physics-constrained RL problem using PPO with a solver-step cap and smoothness penalties, reporting higher fidelities and robustness than prior methods.
Problem
Finding high-fidelity, experimentally realistic control pulses for open (noisy, dissipative) quantum systems, where gradient-based and analytic methods struggle and simulation cost grows with system size and fast dynamics.
Approach
Constrain the RL policy to signals whose quantum simulation fits within a maximum number of adaptive solver steps (a proxy for slow, adiabatic dynamics) and penalise non-smooth/high-area signals via reward shaping, enabling parallel JAX-based optimisation of many agents on one GPU.
Key results
- PPO on the four-level Λ system reaches mean fidelity 0.999 ± 0.0003 averaged over 32 seeds, versus 0.99 for vanilla PPO, 0.985 for vanilla DDPG and 0.99 for Krotov-type baselines in Table 1.
- For the two-photon Rydberg C-Z gate, a 0.25µs pulse reaches F ≈ 0.99958, compared with 0.997 numerical and 0.99 analytic solutions cited.
- For transmon reset with T1 = 500µs, the RL-discovered waveform achieves 0.9997 fidelity, an order of magnitude lower reset error than the 0.983 experimental result of Egger et al. (2018a).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the solver-step constraint and physics-motivated reward (ablation) | PPO, DDPG, and TD3 on the Λ system · Noise-free Λ system | mean fidelity | Helped |
| Use PPO instead of DDPG or TD3 | RL for Λ-system population transfer · Λ system population transfer | fidelity: 0.999 ± 0.0003; 0.985 ± 0.004; 0.625 ± 0.01 | Helped |
| Add Gaussian convolution smoothing to control signals (ablation) | Physics-constrained RL · Quantum control tasks | solution quality | Helped |
| Increase pulse-area penalty (ablation) | Λ-system control signals · Λ system population transfer | fidelity | Hurt |
| Use multi-step feedback instead of the bandit setting (ablation) | RL for Λ-system control under biased noise · Λ system with large noise bias | infidelity: 2x reduction in infidelity | Helped |
| Use the RL-discovered reset waveform instead of a smoothed square pulse | Transmon reset · Transmon reset | reset error: 0.03% | Helped |
| Add a Heaviside detuning correction during pulse ring-down (ablation) | Transmon reset waveform · Transmon reset | fidelity | Helped |
Surprising findings
- Vanilla TD3 performed very poorly (0.625 fidelity) on the Λ system, while PPO was far superior among the RL algorithms tested.
- Constraining the maximum solver steps improved solution quality as well as reducing compute time, rather than trading one off against the other.
Limitations
- Constraining solver steps may exclude fast, non-adiabatic control strategies.
- Method depends on accurate system models; validation on physical hardware is left to future work.
- Only partial coverage of experimental imperfections; fully accounting for all noise is future work.
Experimental setup
Benchmarks: Lambda system population transfer two-photon Rydberg C-Z gate transmon unconditional reset
Metrics: fidelity infidelity (1-F) robustness to noise computational time per RL update pulse area
Baselines: GRAPE (Giannelli et al., 2022b) Krotov Analytic Vasilev et al., 2009 Reinforce (Brown et al., 2021) PINN (Norambuena et al., 2023) Vanilla DDPG Vanilla TD3 Vanilla PPO BFGS
Builds on: PPO DDPG TD3 GRAPE Krotov
Closest papers by meaning
- One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective (NeurIPS 2026 · similarity 0.87)
- Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach (ICML 2025 · similarity 0.86)
- Benchmarking Quantum Reinforcement Learning (ICML 2025 · similarity 0.86)
- Differentiable Quantum Computing for Large-scale Linear Control (NeurIPS 2024 · similarity 0.85)
- TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search (NeurIPS 2025 · similarity 0.85)
- Curriculum reinforcement learning for quantum architecture search under hardware errors (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.