Research map / Large language models / Reasoning and chain of thought
Sample Lottery: Unsupervised Discovery of Critical Instances for LLM Reasoning
Zhiping Xiao, Yusheng Zhao, Qixin Zhang, Jiaye Xie, Wanjia Zhao, Weizhi Zhang, Xiao Luo, Philip S. Yu, Ming Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sample Lottery, an unsupervised method that identifies a small set of critical training instances for RLVR-based LLM reasoning, so that policy optimization need not rely on full annotation or uniform compute over all samples.
Problem
RLVR for LLM reasoning typically requires fully annotated datasets and allocates compute uniformly across all samples, which is inefficient.
Approach
Frames a "lottery sample hypothesis" for policy optimization of LLMs: a large training set contains a small subset of critical instances sufficient for effective RLVR, and discovers them without supervision.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on 8 CONST-selected questions | LLaMA-3.1-8B-Instruct · Math avg | avg@k accuracy: 20.12 -> 28.31 | Improved |
| train on 8 CONST-selected instead of 8 random questions | LLaMA-3.1-8B-Instruct · Math avg | avg@k accuracy: 22.32 -> 28.31 | Improved |
| CONST with 8 annotated questions vs full-dataset annotation | LLaMA-3.1-8B-Instruct · Math avg | avg@k accuracy: 28.03 -> 28.31 | No clear effect |
| CONST with 8 annotated questions vs full dataset | Qwen2.5-Math-7B · Math avg | avg@k accuracy: 55.00 -> 54.94 | No clear effect |
| CONST with 8 questions vs 8 random questions | Qwen2.5-Math-7B · Math avg | avg@k accuracy: 53.52 -> 54.94 | Improved |
| CONST vs best active learning baseline (budget 8) | DeepSeek-R1-Distill-Qwen-1.5B · Math avg | avg@k accuracy: 45.60 -> 48.14 | Improved |
| add conformal prediction set size selection (vs random within cluster) (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 23.01 -> 28.31 | Improved |
| add conformal prediction set size selection (vs entropy) (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 23.30 -> 28.31 | Improved |
| add clustering for diversity (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 25.54 -> 28.31 | Improved |
| add procedural volatility (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 26.38 -> 28.31 | Improved |
| add outcome volatility (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 23.04 -> 28.31 | Improved |
| select 1 question per b clusters vs 2 per b/2 clusters (ablation) | CONST · Math avg, budget 8, LLaMA-3.1-8B-Instruct | avg@k accuracy: 27.60 -> 28.31 | Improved |
Experimental setup
Builds on: RLVR lottery ticket hypothesis
Closest papers by meaning
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.90)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.90)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.89)
- SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning (ICML 2026 · similarity 0.89)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
