Research map / Large language models / Reasoning and chain of thought

CORE: Concept-Oriented Reinforcement for Bridging the Definition–Application Gap in Mathematical Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CORE, a GRPO-based RL framework that uses textbook concept definitions to synthesize concept-aligned quizzes, inject concept snippets into failed rollouts (trajectory replacement), or add a forward-KL term between concept-primed and unprimed policies, improving math concept application.

Problem

LLMs can recite math concepts but fail to apply them on concept-linked exercises (a definition-application gap), and outcome-only RLVR gives little fine-grained conceptual signal.

Approach

Use explicit textbook concept texts as a controllable training signal: replace all-failed GRPO groups with concept-guided trajectories carrying a reward bonus (CR), or distill concept-primed next-token distributions into the unprimed policy via forward KL (KL), while evaluating without concept text at test time.

Key results

• On Qwen2-Math-7B, CORE-KL reaches 55.7 on Textbook vs 46.4 for Vanilla, and CORE-CR reaches 34.5 on OlympiadBench vs 28.7 for Vanilla.
• CORE-CR on DeepSeek-R1-DQ-1.5B improves average from 72.7 to 73.1 across out-of-domain suites; on Llama-3-8B-Inst from 58.1 to 58.9.
• Self-supervised variant using Qwen2-Math-7B-Instruct as generator still improves most benchmarks (e.g., OlympiadBench 28.7 to 32.9).

Surprising findings

• Random-reward GRPO yields no meaningful gains, suggesting the improvements are not from reward noise alone.
• CORE-CR underperforms Vanilla on TabMWP for DeepSeek-R1-DQ-1.5B (87.6 vs 89) and on GSM8K for Qwen2.5-Math-1.5B (57.8 vs 60.7).

Limitations

• In-domain evaluation uses only 140 multiple-choice exercises from a single linear algebra textbook, with gains often within a few points.
• Textbook is translated from Chinese and copyright/ethics constraints limit redistribution; artifacts not clearly released.
• Mechanism analysis relies on manual labeling of 19 problems and a single case study; gains are small on several out-of-domain benchmarks and some regressions occur.

Experimental setup

Datasets: Advanced Algebra (3rd Edition) textbook concepts and exercises (translated) GSM8K ASDiv MAWPS TabMWP MATH MMLU-STEM Gaokao 2023 (EN) GaokaoMathQA CMath Minerva Math SVAMP

Benchmarks: Textbook GSM8K ASDiv MAWPS TabMWP MATH MMLU-STEM Gaokao 2023 (EN) GaokaoMathQA CMath Minerva Math SVAMP

Metrics: accuracy (SC@21, T=0.7) F1 (CounterMATH) RUD_K retention under distractors

Baselines: Vanilla SFT Random-Reward GRPO Top 4 of 6 GRPO BREAD Concept-Verifier RL

Builds on: GRPO BREAD verl

Closest papers by meaning

• MARGE: Improving Math Reasoning with Guided Exploration (ICML 2025 · similarity 0.89)
• AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking (ICLR 2026 · similarity 0.89)
• MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy (AAAI 2026 · similarity 0.88)
• MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model (ICLR 2026 · similarity 0.88)
• MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning (ICLR 2024 · similarity 0.88)
• Probing to Refine: Reinforcement Distillation of LLM Reasoners via Explanatory Inversion (ICLR 2026 · similarity 0.88)