AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

On the Primacy Bias in RLVR Training

Nguyen Phuc, Ngoc-Hieu Nguyen, Parshin Shojaee, Chinh D La, Duy M. H. Nguyen, Rui Zhang, Khoa D Doan, Ting Hua, Nitesh Chawla

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies whether RLVR can extend LLM reasoning beyond the base model by analyzing primacy bias, the over-reliance on already-learned representations.

Problem

Unclear whether reinforcement learning with verifiable rewards (RLVR) can extend reasoning capabilities beyond what is already encoded in the base model.

Approach

Frames the question through the lens of primacy bias, the tendency of models to over-rely on already-learned representations, applied to RLVR training.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train GRPO on hard problems only instead of full dataset (ablation)GRPO · Qwen2.5-3B-Instruct, MinervaPass@128 (%): 48.83 -> 52.98Improved
train GRPO on hard problems only instead of full dataset (ablation)GRPO · Qwen2.5-3B-Instruct, OlympiadBenchPass@128 (%): 67.87 -> 69.87Improved
train GRPO on easy problems only (ablation)GRPO · Qwen2.5-3B-Instruct, MATH-500Pass@128 (%): 94.31 -> 92.47Hurt
train GRPO on hard problems only instead of full dataset (ablation)GRPO · Llama-3.2-3B-Instruct, MATH-500Pass@128 (%): 83.4 -> 89.1Improved
train GRPO on hard problems only vs base (ablation)GRPO · Qwen2.5-3B-Instruct, MATH-500Pass@128 (%): 94.31 -> 94.51No clear effect
prolong hard-only training from 512 to 2048 steps (ablation)GRPO (Hard) · Qwen2.5-3B-Instruct, MATH-500Pass@kHurt
prolong hard-only training (ablation)GRPO (Hard) · Qwen2.5-3B-Instruct, base-unsolvable problemsPass@kImproved
RL fine-tuning vs SFTRLVR models · public checkpointsKL to linearized model: 12.73 vs 0.06Improved
pre-train then RL vs random initialization (ablation)GRPO on MNIST · toy MNISTPass@k coverageHurt
increase pre-training skew beta (ablation)GRPO on MNIST · toy MNISTsteps to learn hard classesHurt

Experimental setup

Builds on: RLVR

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy