Research map / Reinforcement learning / Visual and representation RL

Verifying Neural Networks with Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Scalable RL systems and environments · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RSB, an actor-critic DRL method that learns attention weights to rescale baseline branching heuristic scores (FSB) in neural network verification branch-and-bound, improving solved instances and reducing branches.

Problem

Branching heuristics in branch-and-bound DNN verifiers make greedy decisions with hand-crafted scores that ignore long-term search efficiency and do not leverage verification data.

Approach

Learn an actor-critic policy (PointerNet-based, with GCN graph embeddings and raw neuron features) that outputs per-neuron weights multiplying a baseline heuristic's scores, trained with a reward that penalizes unresolved subproblems so the policy optimizes cumulative future efficiency.

Key results

• RSB solves 165 instances across FNN, CNN and Sigmoid benchmarks versus 148 for FSB, an 11% improvement.
• On FNN, RSB uses 1.86M branches vs 6.65M for FSB (about 72% fewer) and solves 50 vs 43 instances.
• On unseen Sigmoid networks RSB solves 43 instances vs 34 for FSB, with 5.14M vs 8.99M branches.

Surprising findings

• Polarity explores far more branches (~170M on FNN) yet solves the fewest instances, indicating branch count alone does not determine success.
• FSB performs worse than UPB on unseen CNN/Sigmoid architectures.

Limitations

• Improvements over FSB are modest on CNN (72 vs 71 solved) and evaluation uses only a 120s timeout per instance.
• Training data is synthetic FNN instances, and evaluation is on a limited set of 600 instances.

Experimental setup

Datasets: VNN-COMP'25 regular track (motivation statistic) synthetic FNN verification instances

Benchmarks: FNN verification instances (200) CNN verification instances (200) Sigmoid verification instances (200)

Metrics: solved instances explored branches runtime (s) decision time per step

Baselines: FSB UPB Polarity BaBSR

Builds on: FSB BaBSR soft actor-critic PointerNet NeuralSAT

Closest papers by meaning

• Training Verification-Friendly Neural Networks via Neuron Behavior Consistency (AAAI 2025 · similarity 0.86)
• VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees (ICML 2024 · similarity 0.85)
• Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planes (NeurIPS 2024 · similarity 0.85)
• Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification (NeurIPS 2025 · similarity 0.85)
• Verification Learning: Make Unsupervised Neuro-Symbolic System Feasible (ICML 2025 · similarity 0.85)
• VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models (ICLR 2026 · similarity 0.84)