Research map / Reinforcement learning / RL from feedback
TGPO: Efficient Policy Optimization through Sequence Anchor and Information Gating
Hang Ding, Dongqi Liu, Qiming Feng, Jian Li, Tong Lei, Jiafu Wu, Shuo Wang, Jiangning Zhang, Chengjie Wang, Yabiao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sequence anchor + information gate (TGPO) vs best baseline DAPO | TGPO · Avg of 7 math benchmarks | greedy@1: 46.7 -> 50.5 | Improved |
| add TGPO to Llama-3.2-3B-Instruct vs DAPO | TGPO · Avg of 7 math benchmarks | greedy@1: 26.2 -> 27.0 | Improved |
| add TGPO vs DAPO under sampling | TGPO · Avg@32 / Pass@32 | Avg@32, Pass@32: 37.9 -> 38.8; 58.0 -> 62.2 | Improved |
| add sequence anchor (ablation) | TGPO · Qwen2.5-Math-7B ablation avg | greedy@1: 37.9 -> 42.8 | Improved |
| add information gate (ablation) | TGPO · Qwen2.5-Math-7B ablation avg | greedy@1: 36.9 -> 42.8 | Improved |
| add consistency trust term (ablation) | TGPO · Qwen2.5-Math-7B ablation avg | greedy@1: 37.2 -> 42.8 | Improved |
| add drift trust term (ablation) | TGPO · Qwen2.5-Math-7B ablation avg | greedy@1: 39.7 -> 42.8 | Improved |
| use tau=2 rather than 1 or 3 (ablation) | TGPO · Qwen2.5-Math-7B ablation avg | greedy@1: 38.9 (tau=1), 39.1 (tau=3) vs 42.8 | Improved |
| replace GSPO sequence-level weighting with TGPO | GSPO · Qwen2.5-Math-7B ablation avg | greedy@1: 33.8 -> 42.8 | Improved |
| retain imperfect rollouts via down-weighting vs sequence-level discarding | TGPO vs GSPO · training | non-zero gradient fraction | Improved |
| use token-level IS (GRPO) instead of TGPO | GRPO · training dynamics | IS variance, entropy | Hurt |
Closest papers by meaning
- Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.91)
- STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens (NeurIPS 2026 · similarity 0.90)
- Not Only Where, But When: Temporal Scheduling for RLVR (NeurIPS 2026 · similarity 0.90)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.89)
- TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks (NeurIPS 2026 · similarity 0.89)
- Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
