Research map / Large language models / Instruction and fine-tuning
Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning
Jingchu Gai, Guanning Zeng, Huaqing Zhang, Aditi Raghunathan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add differential smoothing (DS-GRPO) to GRPO | GRPO · Pass@K vs base model | Pass@8192 change vs base: -3.80 -> +1.60 | Improved |
| add differential smoothing (DS-GRPO) to GRPO | GRPO · Pass@1 vs base model | Pass@1 change vs base: +9.49 -> +12.73 | Improved |
| add differential smoothing to GRPO | GRPO · Countdown | Pass@K: approx 4% for all K | Improved |
| add differential smoothing to GRPO | GRPO · Math reasoning (3 models, 5 benchmarks) | Pass@1: 0.2%--2.9% | Improved |
| add differential smoothing to GRPO | GRPO · Math reasoning (3 models, 5 benchmarks) | Pass@64: 0.5%--6.7% | Improved |
| add differential smoothing to GRPO | GRPO · Math reasoning | inference samples to match Pass@64: 4x inference speedup (k=16) | Improved |
| add DS-GRPO under varied sampling temperature (ablation) | GRPO · Countdown (temperature sweep) | Pass@K: approx 4% | Improved |
| add DS-GRPO under varied KL coefficient (ablation) | GRPO · Countdown (KL coefficient sweep) | Pass@K: approx 3.2% | Improved |
| modify both correct and incorrect trajectories rather than only correct (add negative modification) (ablation) | DS-GRPO-Positive · Countdown/Math | Pass@K | Improved |
| modify both correct and incorrect trajectories rather than only incorrect (add positive modification) (ablation) | DS-GRPO-Negative · Countdown/Math | Pass@K | Improved |
| use DS-GRPO instead of GR-PKPO (Pass@K reward) | GRPO · Countdown | Pass@K | Improved |
| use DS-GRPO instead of CISPO | GRPO · math reasoning datasets | accuracy | Improved |
Closest papers by meaning
- Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity (ICLR 2026 · similarity 0.91)
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.91)
- SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning (ICML 2026 · similarity 0.90)
- Understanding the Effects of RLHF on LLM Generalisation and Diversity (ICLR 2024 · similarity 0.89)
- Jointly Reinforcing Diversity and Quality in Language Model Generations (NeurIPS 2026 · similarity 0.89)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
