Research map / Reinforcement learning / RL from feedback
Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
Jiaming Li, Longze Chen, Ze Gong, Yukun Chen, Lu Wang, Wanwei He, Run Luo, Minzheng Wang, Lei Zhang, Haoran Ye, Min Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PACS supervised formulation instead of GRPO | Qwen3-8B · BeyondAIME | pass@64: 57.64 -> 64.02 | Improved |
| Use PACS instead of PPO | Qwen3-8B · BeyondAIME | pass@64: 53.80 -> 64.02 | Improved |
| Train Qwen3-4B with PACS | Qwen3-4B · AIME 2025 | pass@1: 34.71 -> 45.63 | Improved |
| Add class-imbalance weighting to PACS (ablation) | PACS Qwen3-8B · BeyondAIME | pass@64: 58.97 -> 64.02 | Improved |
| Add class-imbalance weighting to PACS (ablation) | PACS Qwen3-4B · AMC 23 | pass@1: 89.18 -> 90.45 | Improved |
| Add class-imbalance weighting to PACS (ablation) | PACS Qwen3-8B · AMC 23 and AIME 2024 | pass@1: AIME 2024: 58.93 (w/o weight) vs 57.58 | No clear effect |
| Use RLOO score function instead of GRPO or Dr. GRPO group-mean score (ablation) | PACS · BeyondAIME | pass@1: 26.41 / 26.67 -> 27.16 | Improved |
| Vary beta in {0.1, 0.5, 1, 2, 10} (ablation) | PACS · AMC23, AIME, BeyondAIME | pass@k | No clear effect |
| Train with PACS instead of GRPO or PPO, evaluate out-of-domain | PACS Qwen3-8B · LiveCodeBench | score: 38.01 -> 45.96 | Improved |
| Train with PACS instead of GRPO, evaluate on GPQA | PACS Qwen3-8B · GPQA | score: 48.55 -> 52.46 | Improved |
| Use PACS instead of GRPO | Qwen3-8B · AIME 2025 response diversity | diversity score: 0.0249 -> 0.0392 | Improved |
| Use PACS instead of GRPO with continuous reward model supervision | Qwen3-4B · AIME 2025 | pass@1: 37.08 -> 40.91 | Improved |
Closest papers by meaning
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.90)
- On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation (ICLR 2026 · similarity 0.90)
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories (NeurIPS 2026 · similarity 0.89)
- Generalization of RLVR Using Causal Reasoning as a Testbed (ICLR 2026 · similarity 0.89)
- TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning (ICLR 2026 · similarity 0.89)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
