Research map / Reinforcement learning / Policy gradient and actor-critic

Replay-buffer engineering for noise-robust quantum circuit optimization

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReaPER+ (annealed PER-to-ReaPER replay), OptCRLQAS (amortized quantum-classical evaluation), and weight-free replay-buffer transfer from noiseless to noisy quantum circuit optimization.

Problem

RL-based quantum circuit optimization wastes experience: replay buffers ignore TD-target reliability, curriculum architecture search runs a full quantum-classical evaluation every step, and noiseless trajectories are discarded when retraining under noise.

Approach

Anneal replay priorities from TD-error prioritization (PER) to reliability-aware prioritization (ReaPER) via a linear schedule on the reliability exponent; amortize expensive evaluations over m edits; and copy a noiseless replay buffer into the noisy target buffer without weight transfer.

Key results

• ReaPER+ reaches fidelity 0.9920 on 2-qubit ZZ(pi) in 2.5x10^4 episodes versus 10^5 for PER/ReaPER/HER and 8x10^5 for PPO.
• OptCRLQAS reduces average wall-clock time per episode by 67.5% on 12-qubit H2O relative to CRLQAS.
• Noiseless-to-noisy buffer transfer reduces steps to chemical accuracy by 85-90% and final energy error by up to 90% relative to from-scratch noisy learning.

Surprising findings

• Fixed ReaPER reaches the lowest energy error at 12 qubits while ReaPER+ gets a competitive but slightly higher error, so the annealed rule does not dominate at every scale.
• Buffer transfer's benefit depends on the exploration rate and does not improve final energy accuracy at 12 qubits.

Limitations

• Only DQN/DDQN backbone tested; off-policy actor-critic and on-policy methods not covered.
• Transfer assumes shared state and action spaces; all quantum experiments run on classical simulators with depolarizing or amplitude damping noise, not real hardware.
• Theoretical variance-reduction result is conditional on unverified assumptions (MLR condition).

Experimental setup

Datasets: Haar-random single-qubit target unitaries molecular Hamiltonians (H2, BeH2, H2O, Heisenberg model) LunarLander-v3

Benchmarks: 1-qubit compiling (HRC and small-rotation bases) 2-qubit ZZ(pi) compiling 5-qubit Heisenberg QAS 6-qubit BeH2 QAS 8-qubit H2O QAS 12-qubit H2O QAS LunarLander-v3

Metrics: success probability fidelity steps to chemical accuracy final energy error (Ha) CNOT count gate count wall-clock time AUC

Baselines: PER ReaPER HER uniform replay PPO DQAS GQAS TF-QAS SA-QAS quantumDARTS CRLQAS

Builds on: DQN DDQN prioritized experience replay ReaPER CRLQAS TensorRL-QAS

Closest papers by meaning

• Curriculum reinforcement learning for quantum architecture search under hardware errors (ICLR 2024 · similarity 0.88)
• TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search (NeurIPS 2025 · similarity 0.87)
• Reinforced Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements (ICML 2025 · similarity 0.86)
• AI-Powered Algorithm-Centric Quantum Processor Topology Design (AAAI 2025 · similarity 0.86)
• TITAN: A Trajectory-Informed Technique for Adaptive Parameter Freezing in Large-Scale VQE (NeurIPS 2025 · similarity 0.85)
• One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective (NeurIPS 2026 · similarity 0.84)