Research map / Large language models / Reasoning and chain of thought
Boosting Multi-Domain Reasoning of LLMs via Curvature-Guided Policy Optimization
Xize Liang, Lin Yang, Jie Wang, Rui Liu, Yang Lu, Jinliang Zeng, Hanzhu Chen, Dong Li, Jianye HAO
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a curvature-guided policy optimization approach for multi-domain RL of LLMs to mitigate cross-domain conflicts and improve reasoning across domains.
Problem
Multi-domain RL for LLMs has intricate reward surfaces and cross-domain conflicts, where gains in one capability can degrade another.
Approach
Uses curvature information of the reward surface to guide policy optimization so that parameters can excel across domains; the abstract states the goal but the specific mechanism is not described in the provided text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CGPO instead of joint learning | Qwen2.5-7B-Instruct · Multi-domain average (7 benchmarks) | average score: 56.62 -> 59.59 | Improved |
| use CGPO instead of joint learning | Qwen2.5-3B-Instruct · Multi-domain average (7 benchmarks) | average score: 49.04 -> 50.42 | Improved |
| use CGPO instead of joint learning | Qwen2.5-3B-Instruct · MATH500 | score: 64.50 -> 64.20 | No clear effect |
| use CGPO instead of joint learning | Qwen2.5-3B-Instruct · WritingBench | score: 58.61 -> 63.04 | Improved |
| use CGPO instead of joint learning | Qwen2.5-7B-Instruct · MATH500 | score: 76.00 -> 75.55 | Hurt |
| randomize domain order each update (vs fixed order) (ablation) | CGPO (Qwen2.5-7B-Instruct) · Multi-domain average (7 benchmarks) | average score: 58.48 -> 59.59 | Improved |
| set mixing coefficient alpha to 1.2 rather than 0.9 or 1.5 (ablation) | CGPO (Qwen2.5-7B-Instruct) · Multi-domain average (7 benchmarks) | average score: 58.15 / 58.04 -> 59.59 | Improved |
| add CGPO overhead vs joint learning | Qwen2.5-7B-Instruct · Training time | hours per epoch: 17.8 -> 18.6 | No clear effect |
Closest papers by meaning
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.92)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.90)
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- SABER: Switchable and Balanced Training for Efficient LLM Reasoning (AAAI 2026 · similarity 0.89)
- Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models (NeurIPS 2026 · similarity 0.89)
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
