Research map / Reinforcement learning / Policy gradient and actor-critic

One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Alternative policy update rules · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Casts variational energy minimization for autoregressive neural quantum states as an advantage policy-gradient problem and introduces PWO, a PPO-style clipped trust-region optimizer with amplitude and phase clipping, outperforming Adam, minSR and SPRING on spin chains and fine-tuning a 1.5B RWKV-7 model.

Problem

First-order optimization of autoregressive neural quantum states (NQS) for ground-state search is unstable or inaccurate, while stochastic reconfiguration variants are costly and numerically fragile at scale.

Approach

Views the VMC energy gradient as an advantage-weighted policy gradient over the Born distribution and applies PPO-style clipped probability-ratio surrogates for amplitudes plus a clipped wrapped-phase-increment surrogate for phases, allowing sample reuse with first-order updates.

Key results

• PWO reaches relative error 1e-7 on the 1D Ising model in about 5 minutes, versus about 30 minutes for minSR.
• On the Heisenberg J1-J2 chain, minSR produces NaNs in 6 of 10 runs, and minSR and SPRING remain at relative error about 1e-1 after 30 minutes while PWO reaches 1e-7 in about 15 minutes.
• PWO fine-tunes a 1.5B-parameter RWKV-7 model as an NQS, reaching lower final relative error and V-score than Adam.

Surprising findings

• minSR is numerically unstable on the J1-J2 chain, with 6 of 10 runs producing NaNs.

Limitations

• Derivation of the policy-gradient form assumes stoquastic Hamiltonians with fixed phase for the amplitude channel.
• Theoretical improvement certificate requires global trust-region constraints that the clipping does not guarantee on all configurations.
• Large-model experiments are limited to 1D Ising with RWKV-7, and 2D results only compare against Adam.

Experimental setup

Benchmarks: 1D transverse-field Ising chain N=12 1D Heisenberg J1-J2 chain N=12 2D J1-J2 square lattice 10x10 RWKV-7 fine-tuning on 1D Ising

Metrics: relative energy error V-score wall-clock time to accuracy NaN frequency

Baselines: Adam minSR SPRING ISGO

Builds on: PPO TRPO Stochastic reconfiguration minSR SPRING

Closest papers by meaning

• PALQO: Physics-informed model for Accelerating Large-scale Quantum Optimization (NeurIPS 2025 · similarity 0.89)
• TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search (NeurIPS 2025 · similarity 0.88)
• Q-MAML: Quantum Model-Agnostic Meta-Learning for Variational Quantum Algorithms (AAAI 2025 · similarity 0.87)
• Neural Quantum Spectral Operator Learning for Solving Partial Differential Equations (NeurIPS 2026 · similarity 0.87)
• Rethink the Role of Deep Learning towards Large-scale Quantum Systems (ICML 2025 · similarity 0.87)
• Score-based Variational Inference via Quantum Maximally Mixed States (NeurIPS 2026 · similarity 0.87)