Research map / Reinforcement learning / RL from feedback
Group-Normalized Implicit Value Optimization for Language Models
Yunseon Choi, Junyoung Jang, Chaeyoung Oh, Minchan Jeong, Doo Hwan Hwang, Kee-Eung Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| sequential step-level objective instead of one-step variant (ablation) | GN-IVO · Helpful assistant (Llama-3.2-3B-Instruct) | Avg@1: 2.562 -> 3.370 | Improved |
| sequential step-level objective instead of one-step variant (ablation) | GN-IVO · Helpful assistant (Qwen2.5-1.5B-Instruct) | Avg@1: 1.446 -> 1.650 | Improved |
| sequential step-level objective instead of one-step variant (ablation) | GN-IVO · TL;DR (Llama-3.2-3B-Instruct) | Avg@1: 3.398 -> 3.347 | No clear effect |
| one-step variant of GN-IVO vs GRPO and RLOO (ablation) | GN-IVO one-step · Text generation geometric mean (Llama-3.2-3B-Instruct) | GM: 1.154 -> 1.478 | Improved |
| increase group size K (ablation) | GN-IVO and GRPO · HH-RLHF (Qwen2.5-1.5B-Instruct) | reward | Improved |
| small group size K=2 (ablation) | GN-IVO vs GRPO · HH-RLHF (Qwen2.5-1.5B-Instruct) | reward | Improved |
| lower temperature alpha 0.1 or 0.2 instead of 0.5 or 1.0 (ablation) | GN-IVO · HH-RLHF (Qwen2.5-1.5B-Instruct) | reward: alpha 0.5, 1.0 -> 0.1, 0.2 | Improved |
| GN-IVO vs GRPO | GRPO · AIME2024 (Qwen2.5-Math-7B) | Pass@1: 26.6 -> 30.0 | Improved |
| GN-IVO vs GRPO | GRPO · Olympiad-Bench (Llama-3.1-8B-Instruct) | Pass@1: 18.8 -> 17.3 | Hurt |
| GN-IVO vs RLOO | RLOO · AMC2023 (Llama-3.1-8B-Instruct) | Pass@1: 35.0 -> 42.5 | Improved |
| use explicit value networks (PPO, DRO, OREO) | RL baselines · math reasoning | Pass@1/Pass@3 | No clear effect |
Closest papers by meaning
- GVPO: Group Variance Policy Optimization for Large Language Model Post-Training (NeurIPS 2025 · similarity 0.89)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.87)
- Alignment of Large Language Models with Constrained Learning (NeurIPS 2025 · similarity 0.87)
- Amortizing intractable inference in large language models (ICLR 2024 · similarity 0.87)
- Vanishing Gradients in Reinforcement Finetuning of Language Models (ICLR 2024 · similarity 0.87)
- Controlled Decoding from Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
