Research map / Large language models / Reasoning and chain of thought

Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PivotTrace, a label-free three-way data triage for semi-supervised RLVR that counts attention-peak "metacognitive pivots" as an uncertainty proxy to decide which unlabeled questions to annotate, use unsupervised, or discard.

Problem

RLVR needs costly ground-truth annotations and heavy training over large datasets; existing data selection requires labels and unsupervised RLVR underperforms, so the goal is to choose which unlabeled samples to annotate and train on before any labels are available.

Approach

Detect metacognitive pivots as long-range attention peaks in the CoT (top-attention heads, future-window averaging), use pivot count as a label-free proxy for expected correctness, and calibrate thresholds via sliding-window accuracy on a small annotated probing set to route samples into annotate / unlabeled-train / discard sets.

Key results

• With 29.3% annotated and 57.9% of data retained, PivotTrace reaches 49.5 ID and 64.9 OOD average on Qwen3-4B-Base, surpassing fully supervised training (49.2 ID / 64.3 OOD).
• PivotTrace reaches 49.2% average in 160 training steps versus 440 for fully supervised, a claimed 2.75x convergence speedup.
• On Deepseek-R1-Distill-Llama-8B, PivotTrace beats the strongest baseline by 3.1% average ID accuracy and 1.9% OOD.

Surprising findings

• Training on low-uncertainty (high-pass-rate) samples can degrade performance, since they yield near-zero policy gradients.
• Short-range attention heads show minimal selectivity, while long-range heads separate pivot-rich questions much better.

Limitations

• Threshold calibration requires annotating a small probing set (N=100 questions with multiple rollouts).
• Experiments limited to math-focused training data and models up to 8B.
• Pivot detection relies on attention-map extraction over full CoTs, which may be costly for very long responses.

Experimental setup

Datasets: DAPO-Math-14k AIME 2024 AIME 2025 AMC MATH-500 Minerva OlympiadBench ARC-c GPQA-diamond MMLU-Pro

Benchmarks: AIME 2024 AIME 2025 AMC MATH-500 Minerva OlympiadBench ARC-c GPQA-diamond MMLU-Pro

Metrics: avg@32 avg@4 pass@1 average accuracy training steps to threshold

Baselines: Random Consistency Entropy Self-Certainty CoE CoT-Kinetics

Builds on: GRPO TraPO DAPO-Math-14k verl

Closest papers by meaning

• Detecting RLVR Training Data via Structural Convergence of Reasoning (NeurIPS 2026 · similarity 0.91)
• Generalization of RLVR Using Causal Reasoning as a Testbed (ICLR 2026 · similarity 0.90)
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.90)
• On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation (ICLR 2026 · similarity 0.89)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.89)
• TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning (ICLR 2026 · similarity 0.89)