Research map / Reinforcement learning / RL from feedback
SSVPO: Effective Step-Level Credit Assignment for RL Training of Language Models
Yugu Li, Zehong Cao, Jianglin Qiao, Siyi Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with SSVPO instead of vanilla | Qwen3-4B · Seven math benchmarks | average accuracy: 82.1 -> 86.7 | Improved |
| use SSVPO instead of RLOO | Qwen3-4B · Seven math benchmarks | average accuracy: 84.9 -> 86.7 | Improved |
| use SSVPO instead of GRPO | Qwen3-4B · Seven math benchmarks | average accuracy: 83.4 -> 86.7 | Improved |
| use SSVPO instead of DAPO | Qwen3-4B · Seven math benchmarks | average accuracy: 83.0 -> 86.7 | Improved |
| train with SSVPO | Qwen3-4B · Seven math benchmarks | token usage: 6327 -> 5178 | Improved |
| use SSVPO instead of RLOO | Qwen3-4B · GSM8K | tokens per question: 1665 -> 1320 | Improved |
| use SSVPO instead of RLOO | Qwen3-4B · SVAMP | accuracy: 94.8 -> 94.6 | Hurt |
| use SSVPO instead of SPO | DeepSeek-R1-Distill-Qwen-1.5B · Seven math benchmarks | average accuracy: 66.3 -> 68.7 | Improved |
| use SSVPO instead of SPO | DeepSeek-R1-Distill-Qwen-1.5B · Seven math benchmarks | token usage: 5402 -> 5838 | Hurt |
| use SSVPO instead of SPO | DeepSeek-R1-Distill-Qwen-1.5B · AIME 2024 | accuracy: 19.3 -> 28.8 | Improved |
| increase reordered steps from 2 to 4 (ablation) | SSVPO · AMC 2023 | accuracy: 77.5 -> 82.5 | Improved |
| increase reordered steps from 2 to 4 (ablation) | SSVPO · GSM8K | accuracy: 90.5 -> 91.8 | Improved |
Closest papers by meaning
- VinePPO: Refining Credit Assignment in RL Training of LLMs (ICML 2025 · similarity 0.92)
- SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning (ICML 2026 · similarity 0.90)
- Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models (NeurIPS 2025 · similarity 0.90)
- Reducing Credit Assignment Variance via Counterfactual Reasoning Paths (NeurIPS 2026 · similarity 0.90)
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.90)
- Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
