Research map / Reinforcement learning / RL from feedback
Gradient-Aware Scheduling: Coupling Curriculum and Staleness for Async Reinforcement Learning
Xinyu Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add curriculum-staleness coupling (CSC) (ablation) | GAS · Code (HumanEval + synthetic) | Pass@1 (%): 42.1 -> 60.1 (-18.0 without CSC) | Improved |
| add execution-aware staleness budgets (EAAS) (ablation) | GAS · Code (HumanEval + synthetic) | Pass@1 (%): 55.3 -> 60.1 (-4.8 without EAAS) | Improved |
| add execution-aware staleness budgets (EAAS) (ablation) | GAS · Code (HumanEval + synthetic) | throughput (samples/s): 16.8 -> 22.4 | Improved |
| add bandit curriculum (ACB) (ablation) | GAS · Code (HumanEval + synthetic) | Pass@1 (%): 46.9 -> 60.1 (-13.2 without ACB) | Improved |
| add CSC (discarding stale experiences) (ablation) | GAS · Code (HumanEval + synthetic) | throughput (samples/s): 23.5 -> 22.4 | Hurt |
| use asynchronous GRPO instead of synchronous | GRPO · Code (HumanEval + synthetic) | Pass@1 (%): 39.7 -> 31.8 Pass@1; 9.7 -> 24.3 throughput | Mixed |
| add uniform staleness control to async GRPO | Async-GRPO · Code (HumanEval + synthetic) | Pass@1 (%): 31.8 -> 40.3 | Improved |
| add difficulty-aware coupling instead of uniform staleness control | Async-GRPO · Code (HumanEval + synthetic) | Pass@1 (%): 40.3 -> 60.1 | Improved |
| add curriculum (CCCS) to synchronous GRPO | Sync-GRPO · Code (HumanEval + synthetic) | Pass@1 (%): 39.7 -> 51.5 | Improved |
| scale GAS to Qwen3.5-9B vs sync baseline | GAS · HumanEval (Qwen3.5-9B) | training reward: 0.279 -> 0.430 | Improved |
| GAS vs plain async GRPO at 9B | GAS · HumanEval (Qwen3.5-9B) | training reward: 0.196 -> 0.430 | Improved |
| transfer GAS to GSM8K with same hyperparameters | GAS · GSM8K (Qwen3.5-9B) | training reward: 0.312 -> 0.572 | Improved |
Closest papers by meaning
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.89)
- AsyncOPD: How Stale Can On-Policy Distillation Be? (NeurIPS 2026 · similarity 0.89)
- Not Only Where, But When: Temporal Scheduling for RLVR (NeurIPS 2026 · similarity 0.88)
- Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis (ICLR 2025 · similarity 0.88)
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training (NeurIPS 2026 · similarity 0.88)
- Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
