Research map / Large language models / Reasoning and chain of thought

Detecting RLVR Training Data via Structural Convergence of Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Min-k NN Distance, a black-box detector that flags prompts seen during RLVR training by the lower nearest-neighbor edit distance among multiple sampled completions, since RLVR makes seen-prompt outputs more structurally rigid.

Problem

Detecting whether a given prompt was used in RLVR training of a reasoning model, where likelihood-based membership inference methods are less effective because RLVR optimizes on self-generated reward-scored trajectories rather than fixed text.

Approach

RLVR induces structural convergence of reasoning trajectories for seen prompts; the average of the k smallest nearest-neighbor normalized edit distances among sampled completions serves as a sampling-only membership score, with lower values indicating RLVR exposure.

Key results

• Min-k NN Distance achieves average AUC 0.70 across eight RLVR-trained models, a 17% relative improvement over the strongest baseline (PPL, 0.60).
• On the DAPO-Qwen-32B setting, Min-k NN Distance reaches AUC 0.72 on original prompts and 0.71 on paraphrased prompts.
• On OpenR1-Distill-7B distillation prompts, Min-k NN Distance achieves AUC 0.76 versus 0.70 for PPL.

Surprising findings

• Likelihood-based baselines such as PPL perform near random or inconsistently on several RLVR models, with Min-K% at 0.42 average AUC, below chance.
• Higher sampling temperature strengthens the detection signal, the opposite of what might be expected from reduced variability.

Limitations

• Requires generating multiple completions per prompt, raising inference cost.
• Detection is weaker on code data than math (AUC 0.69 vs 0.80), attributed to higher diversity in coding tasks.
• Relies on the assumption that RLVR-induced structural collapse is observable across models and tasks.

Experimental setup

Datasets: DAPO dataset RL-MIA K&K RL-MIA SAT AIME 2024 AIME 2025 Beyond-AIME Omni-Math MATH-500 GSM8K Eurus Coding OpenR1-Distill

Benchmarks: RL-MIA K&K RL-MIA SAT open-source RLVR models (DAPO-Qwen-32B, JustRL, ORZ-7B, SimpleRL-32B) OpenR1-Distill distillation prompts ProRL math and code (Nemotron-Research-Reasoning-Qwen-1.5B) GSM8K dual-stage contamination

Metrics: AUC latency

Baselines: PPL Min-K% Min-K%++ CDD Recall Self-Critique

Builds on: Min-K% Prob CDD Self-Critique DAPO GRPO

Closest papers by meaning

• On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation (ICLR 2026 · similarity 0.92)
• The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.92)
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.92)
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.91)
• Generalization of RLVR Using Causal Reasoning as a Testbed (ICLR 2026 · similarity 0.91)
• The Unlearnability Phenomenon in RLVR for Language Models (ICML 2026 · similarity 0.91)