Research map / Reinforcement learning / RL from feedback
Jackpot: Align Actor-Policy Distribution for scalable and stable RL for LLM
Zhuoming Chen, Hongyi Liu, Yang Zhou, Haizhong Zheng, Beidi Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Jackpot rejection sampling for 64x rollout/train batch mismatch | Qwen3-4B-Base RL · AMC22 & 23 (2048/32) | Mean@4: 39.15 -> 53.92 (vs off-policy) | Improved |
| Jackpot vs TIS + Adjustment under extreme 128x mismatch | Qwen3-4B-Base RL · MATH-500 | Mean@4: 19.10 -> 80.00 | Improved |
| Jackpot vs TIS + Adjustment at 64x | Qwen3-4B-Base RL · GSM8K | Mean@4: 92.76 -> 92.24 | No clear effect |
| Jackpot vs TIS + Adjustment at 64x | Qwen3-4B-Base RL · AMC22 & 23 | Mean@4: 57.22 -> 53.92 | Hurt |
| Jackpot vs TIS + Adjustment at 64x | Qwen3-8B-Base RL · AMC22 & 23 | Mean@4: 60.54 -> 63.55 | Improved |
| add Jackpot to separate rollout and trainer models (Qwen2.5-1.5B-IT rollout, 3B-Base trainer) | GRPO · MATH-500 and AMC | accuracy: up to 12% on MATH-500 | Improved |
| add reweighting on top of rejection masking under staleness (ablation) | Jackpot · BF16 staleness, AMC | pass@1: 49.699 -> 63.855 | Improved |
| apply rejection sampling alone under FP8 on-policy training (ablation) | Jackpot masking-only · FP8 on-policy, AIME24 | pass@1: 23.958 -> 25.625 | Improved |
| vary truncation constant C1 in 2 to 8 (ablation) | Jackpot · benchmarks | pass@1 | No clear effect |
| increase top-k from 10 to 20 or 40 for Z estimate (ablation) | Jackpot · AMC, MATH500 | pass@1: 61.446 -> 63.855 -> 61.446 (AMC) | No clear effect |
| raise rejection threshold above 1.0 (ablation) | Jackpot · AMC | pass@1: 48.5 -> 45.8 (c=1.2) | No clear effect |
Closest papers by meaning
- ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models (ICML 2024 · similarity 0.89)
- Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models (ICLR 2024 · similarity 0.89)
- Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs (ICLR 2026 · similarity 0.89)
- Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening (ICML 2026 · similarity 0.89)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.89)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
