Research map / Large language models / Reasoning and chain of thought

V1: Unifying Generation and Self-Verification for Parallel Reasoners

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes V1, a framework pairing uncertainty-guided tournament-based pairwise self-verification at inference (V1-Infer) with an online RL method that co-trains one model as generator and pairwise self-verifier (V1-PairRL).

Problem

Parallel reasoning (sampling N solutions and selecting one) is bottlenecked by self-verification; pointwise scalar scoring of candidates by LLMs is poorly calibrated, and self-aggregation can cause diversity collapse. The paper asks how to self-verify candidates pairwise efficiently and how to train

Approach

Replace pointwise self-scoring with pairwise self-verification where the same model rates two solutions (1-10), aggregated via magnitude-weighted win rates and allocated by a two-phase Swiss-style tournament that focuses comparisons on uncertain near-tie pairs; and co-train generation and pairwise verification online within GRPO-style RL using a sparsity-thresholded verifier reward and correct-con

Key results

• V1-Infer improves Pass@1 by up to 10% over pointwise verification (e.g., GPT-OSS-20B on HMMT +10.0%, CodeContests 66.06% to 73.33%).
• V1-PairRL achieves +6.5% on LiveCodeBench-v5 over V1-PointRL at 2x budget (53.9% vs 47.4%) and +8.9% on CodeContests over RL baseline under V1-Infer.
• On SWE-bench Lite with Gemini-2.5-Flash (N=8), pairwise verification reaches 33.3% resolve rate vs 28.3% pointwise and 26.3% vanilla.

Measured effects reported in the paper

Surprising findings

• Recursive self-aggregation shows diversity collapse: Pass@N monotonically decreases with aggregation steps, so refined Pass@1 rarely exceeds initial Pass@N.
• Without a sparsity threshold on the verifier reward, the verifier collapses to safe middle-ground scores (v=0.5) for every solution.

Limitations

• RL training is instantiated only for code generation on one model (Qwen3-4B-Instruct-2507).
• Trained model results use 3 seeds and budget exploration limited to 1x and 2x due to compute cost.
• Verifier reward requires ground-truth correctness labels from unit tests, limiting applicability to verifiable domains.

Experimental setup

Datasets: DeepCoder training set DeepCoder validation set LiveCodeBench-v5 LiveCodeBench-v6 CodeContests SWE-bench Lite AIME 2025 HMMT 2025

Benchmarks: LiveCodeBench-v5 LiveCodeBench-v6 CodeContests SWE-bench Lite AIME 2025 HMMT 2025

Metrics: Pass@1 Pass@N resolve rate accuracy vs. verification budget

Baselines: pointwise self-verification Recursive Self-Aggregation (RSA) standard RL (GRPO-style) V1-PointRL random pairwise verification vanilla first-candidate selection

Builds on: GRPO DAPO Dr. GRPO Recursive Self-Aggregation DeepCoder

Closest papers by meaning

• Variation in Verification: Understanding Verification Dynamics in Large Language Models (ICLR 2026 · similarity 0.91)
• What If We Allocate Test-Time Compute Adaptively? (ICML 2026 · similarity 0.90)
• Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification (ICML 2025 · similarity 0.89)
• SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling (AAAI 2026 · similarity 0.89)
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (NeurIPS 2025 · similarity 0.89)
• TrimR: Verifier-based Training-Free Thinking Trimming for Efficient Test-Time Scaling (ICLR 2026 · similarity 0.88)