Research map / Large language models / Agents, code and math
SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents SynLogic, a framework and 35-task dataset of verifiable synthetic logic puzzles with adjustable difficulty, used for GRPO/DAPO-style RLVR on Qwen2.5 7B and 32B models and mixed with math and coding data.
Problem
Open-source RL reasoning efforts focus on math and coding; general logical reasoning lacks diverse, verifiable, difficulty-controllable training data.
Approach
Build a rule-based generator and verifier for 35 logic tasks with tunable difficulty parameters, calibrated by upper and lower difficulty bounds from strong and chat models, and use the resulting binary-reward data for zero-RL training alone and mixed with math and code.
Key results
- SynLogic-7B reaches 48.1% on KOR-Bench vs 38.6% for Qwen2.5-7B-Instruct, and 44.4% on SynLogic-Val vs 9.0% for the instruct model.
- SynLogic-32B reaches 25.5% on BBEH vs 19.2% for DeepSeek-R1-Distill-Qwen-32B, while scoring 62.2% on KOR-Bench vs 66.6% for the distill model.
- Zero-Mix-3 (SynLogic+Math+Coding) on 32B reaches 28.6 BBEH, 65.0 KOR-Bench, 57.5 GPQA-Diamond, versus 18.5, 58.6, 55.2 for Zero-Mix-2 (Math+Coding).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL-train Qwen2.5-7B-Base on SynLogic-Easy | Qwen2.5-7B-Base · KOR-Bench | accuracy: 11.6 -> 48.1 | Helped |
| RL-train Qwen2.5-32B-Base on SynLogic-Hard | Qwen2.5-32B-Base · BBEH | accuracy: 3.3 -> 25.5 | Helped |
| RL-train Qwen2.5-7B-Base on SynLogic-Easy | Qwen2.5-7B-Base · MATH 500 | accuracy: 64.6 -> 71.8 | Helped |
| RL-train Qwen2.5-32B-Base on SynLogic-Hard | Qwen2.5-32B-Base · MATH 500 | accuracy: 68.6 -> 82.0 | Helped |
| Add SynLogic data to math-and-coding RL training (ablation) | Qwen2.5-32B-Base Zero-Mix-2 · BBEH | pass@1: 18.5 -> 28.6 | Helped |
| Add SynLogic data to math-and-coding RL training (ablation) | Qwen2.5-32B-Base Zero-Mix-2 · KOR-Bench | avg@4: 58.6 -> 65.0 | Helped |
| Add SynLogic data to math-and-coding RL training (ablation) | Qwen2.5-32B-Base Zero-Mix-2 · LiveCodeBench | avg@4: 39.5 -> 40.7 | Helped |
| Add SynLogic data to math-and-coding RL training (ablation) | Qwen2.5-32B-Base Zero-Mix-2 · GPQA-Diamond | avg@4: 55.2 -> 57.5 | Helped |
| Mix SynLogic-Easy with math data instead of math-only training (ablation) | Qwen2.5-7B-Base · Mathematics benchmarks | average accuracy | Helped |
| Mix SynLogic-Easy with math data instead of math-only training (ablation) | Qwen2.5-7B-Base · KOR-Bench | accuracy: nearly 10 absolute percentage points | Helped |
| Mix SynLogic-Easy with coding data instead of coding-only training (ablation) | Qwen2.5-7B-Base · Coding benchmarks | average accuracy | Helped |
| Mix SynLogic-Easy with coding data instead of coding-only training (ablation) | Qwen2.5-7B-Base · KOR-Bench | accuracy: 10 absolute points | Helped |
Surprising findings
- Mixing logic data with math training improved KOR-Bench by roughly 10 points while maintaining math performance under the same step count, consuming fewer math samples.
- Training on purely logic data produced strong math gains (e.g., AMC 2023 +25 points for 7B) without any math training data.
Limitations
- Eight tasks yielded zero accuracy for the 7B model and were removed from the Easy version.
- Only 2 of 35 tasks use existing data; the rest were implemented independently, so fidelity to original benchmarks is not guaranteed.
- The 32B model trails DeepSeek-R1-Distill-Qwen-32B on KOR-Bench and AIME 2024 in the main table.
Experimental setup
Datasets: SynLogic DAPO-Math-17k Codeforces-style coding problems (about 9k, assembled) Zebra Puzzle ARC-AGI
Benchmarks: SynLogic-Val KOR-Bench BBH BBEH AIME 2024 MATH 500 AMC 2023 LiveCodeBench GPQA-Diamond
Metrics: avg@8 pass@1 avg@4 pass@8
Baselines: Qwen2.5-7B-Instruct Qwen2.5-32B-Instruct DeepSeek-R1-Distill-Qwen-32B DeepSeek-R1-Zero-Qwen-32B Zero-Mix-2 (Math+Coding)
Builds on: GRPO DAPO DeepSeek-R1 KOR-Bench BBEH
Closest papers by meaning
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.90)
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.90)
- General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.90)
- RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.89)
- AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.