Research map / Large language models / Reasoning and chain of thought

SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SATURN, a curriculum RL framework that trains LLMs with GRPO on programmatically generated, rule-verifiable SAT instances of increasing difficulty, and releases the SATURN-2.6k benchmark.

Problem

Existing RL tasks for LLM reasoning (math, code, logic puzzles) lack scalability, automatic verifiability, and fine-grained difficulty control needed for curriculum learning.

Approach

Use Boolean SAT instances as RL tasks: they can be generated at scale with known satisfying assignments, verified by rule, and difficulty is tuned via (n,k,l) parameters, with an analytical difficulty estimator D(n,k,l) driving an adaptive pass@1-threshold curriculum.

Key results

• On SATURN-2.6k unseen harder levels, pass@3 average improves by +14.0 (1.5B: 10.1 to 24.2) and +28.1 (7B: 36.1 to 64.2).
• Average across AIME, AMC, MATH-500, GPQA-D, LiveCodeBench rises from 43.4 to 48.2 for SATURN-1.5B and from 61.7 to 63.4 for SATURN-7B.
• On Qwen2.5-7B-Instruct-1M, SATURN with 1k examples reaches average 36.2 vs 33.3 for Logic-RL with 5k examples (+8.8% relative to Logic-RL).

Measured effects reported in the paper

Surprising findings

• SFT on math data drops LiveCodeBench performance (16.4 to 14.6), while SAT RL improves all benchmarks.
• Gains on math and programming plateau with additional curriculum iterations; Iteration-3 on 1.5B lowers average from 48.2 to 46.9.

Limitations

• Gains on math and programming plateau over curriculum iterations, attributed to knowledge limits, context-window bottlenecks, and forgetting.
• The difficulty estimator's R^2 is only about 0.5 for SATURN-7B on easy problems, and the formula is heuristic with an approximate derivation.
• Evaluated only on 1.5B and 7B distilled models; larger models left to future work.

Experimental setup

Datasets: SATURN-2.6k (generated SAT instances) MATH Level-5 training problems (SFT baseline) DeepScaleR 40k math/programming data (prior training of DeepScaleR)

Benchmarks: SATURN-2.6k AIME 24/25 AMC 22/23 MATH-500 GPQA Diamond LiveCodeBench v4_v5

Metrics: pass@k pass@1

Baselines: DeepSeek-R1-Distill-Qwen-1.5B DeepSeek-R1-Distill-Qwen-7B Still-3-1.5B-Preview s1.1-1.5B s1.1-7B z1-7B OpenThinker-7B DeepScaleR-1.5B-Preview Logic-RL ScaleQuest

Builds on: GRPO DeepSeek-R1 Logic-RL OpenRLHF

Closest papers by meaning

• QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation (ICLR 2026 · similarity 0.91)
• Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning (ICLR 2026 · similarity 0.90)
• RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.90)
• SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.90)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.90)